{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 60分钟入门深度学习工具-PyTorch(四、训练一个分类器)\n",
    "**作者**：Soumith Chintala\n",
    "\n",
    "\n",
    "原文翻译自：https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html\n",
    "    \n",
    "中文翻译、注释制作：黄海广\n",
    "\n",
    "github：https://github.com/fengdu78\n",
    "\n",
    "代码全部测试通过。\n",
    "\n",
    "配置环境：PyTorch 1.0，python 3.6，\n",
    "\n",
    "主机：显卡：一块1080ti；内存：32g（注：绝大部分代码不需要GPU）\n",
    "![公众号](images/gongzhong.jpg)\n",
    "### 目录\n",
    "* 1.[Pytorch是什么？](60分钟入门PyTorch-1.PyTorch是什么？.ipynb)\n",
    "* 2.[AUTOGRAD](60分钟入门PyTorch-2.AUTOGRAD.ipynb)\n",
    "* 3.[神经网络](60分钟入门PyTorch-3.神经网络.ipynb)\n",
    "* 4.[训练一个分类器](60分钟入门PyTorch-4.训练一个分类器.ipynb)\n",
    "* 5.[数据并行](60分钟入门PyTorch-5.数据并行.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 四、训练一个分类器"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "你已经学会如何去定义一个神经网络,计算损失值和更新网络的权重。\n",
    "\n",
    "你现在可能在思考：数据哪里来呢？\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 关于数据\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "通常，当你处理图像，文本，音频和视频数据时，你可以使用标准的Python包来加载数据到一个numpy数组中.然后把这个数组转换成`torch.*Tensor`。\n",
    "* 对于图像,有诸如Pillow,OpenCV包等非常实用\n",
    "* 对于音频,有诸如scipy和librosa包\n",
    "* 对于文本,可以用原始Python和Cython来加载,或者使用NLTK和SpaCy\n",
    "对于视觉,我们创建了一个`torchvision`包,包含常见数据集的数据加载,比如Imagenet,CIFAR10,MNIST等,和图像转换器,也就是`torchvision.datasets`和`torch.utils.data.DataLoader`。\n",
    "\n",
    "这提供了巨大的便利,也避免了代码的重复。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在这个教程中,我们使用CIFAR10数据集,它有如下10个类别:’airplane’,’automobile’,’bird’,’cat’,’deer’,’dog’,’frog’,’horse’,’ship’,’truck’。这个数据集中的图像大小为3\\*32\\*32,即,3通道,32\\*32像素。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![cifar10](images/cifar10.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 训练一个图像分类器"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们将按照下列顺序进行:\n",
    "* 使用`torchvision`加载和归一化CIFAR10训练集和测试集.\n",
    "* 定义一个卷积神经网络\n",
    "* 定义损失函数\n",
    "* 在训练集上训练网络\n",
    "* 在测试集上测试网络"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. 加载和归一化CIFAR0\n",
    "使用`torchvision`加载CIFAR10是非常容易的。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "import torchvision\n",
    "import torchvision.transforms as transforms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "torchvision的输出是[0,1]的PILImage图像,我们把它转换为归一化范围为[-1, 1]的张量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Files already downloaded and verified\n",
      "Files already downloaded and verified\n"
     ]
    }
   ],
   "source": [
    "transform = transforms.Compose(\n",
    "    [transforms.ToTensor(),\n",
    "     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])\n",
    "\n",
    "trainset = torchvision.datasets.CIFAR10(root='./data', train=True,\n",
    "                                        download=True, transform=transform)\n",
    "trainloader = torch.utils.data.DataLoader(trainset, batch_size=4,\n",
    "                                          shuffle=True, num_workers=2)\n",
    "\n",
    "testset = torchvision.datasets.CIFAR10(root='./data', train=False,\n",
    "                                       download=True, transform=transform)\n",
    "testloader = torch.utils.data.DataLoader(testset, batch_size=4,\n",
    "                                         shuffle=False, num_workers=2)\n",
    "\n",
    "classes = ('plane', 'car', 'bird', 'cat',\n",
    "           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')\n",
    "#这个过程有点慢，会下载大约340mb图片数据。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们展示一些有趣的训练图像。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAB6CAYAAACvHqiXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztfWmMJdd13nfr7e/13j3T07OvJEVRCxfRtCjLtGRZlC2J\nTgwoEuyEtmkxBuTEDgzEcvzDoGEEBhIYtgM5AWE5ZmJFsiJRoqRIlm1KNLVSJMVtyOEsnOlZet/e\n69dvr3o3P865dc7rbTZqerp1P4Cc6lv1bt1bdavqnPOdxVhr4eHh4eGx+RFs9AA8PDw8PF4f+Be6\nh4eHxxaBf6F7eHh4bBH4F7qHh4fHFoF/oXt4eHhsEfgXuoeHh8cWgX+he3h4eGwRXNUL3RhzrzHm\nuDHmlDHm46/XoDw8PDw8Lh/mSgOLjDEJACcAvAfABQBPA/iItfaV1294Hh4eHh6XiuRV/PZOAKes\ntacBwBjzGQD3AVjzhZ7P521fX99VnNLDw8Pjxw8TExOz1tptFzvual7ouwCcV39fAPAT6/2gr68P\nDz744FWc0sPDw+PHDw899NDZSznuR06KGmMeNMY8Y4x5plqt/qhP5+Hh4fFji6t5oY8B2KP+3s1t\nHbDWPmytvcNae0c+n7+K03l4eHh4rIereaE/DeCIMeaAMSYN4MMAvvT6DMvDw8PD43JxxTZ0a21o\njPktAF8HkADw19baly+3n+PPEofav03s/b2DOwAAbZi4LYpCAEA2kQAAdGWzK/pKpOT7dOtttwAA\nbr/zNul32yBvtXkS6sfG/VbOGRiDaw3ndWQR6tYVx/3RQ/+54+9jZ5+Jt9NJuq3zpUrcVuPrl0un\n4rawWQcA5HsKAIBmJOcsFHpWnDqRoD/aUTtuq9dCHje1FXKZeF+O71Gj2ojbGjYCAETtSM6VzdG4\nM2kAQCtUx/MYbVvubSpNxwdGBpfge5XMkBbYasq9azUqfLz0GzVpvDftlfXh8Iv3fBCAXglAEND5\nTaBkIL4eCKivQK+d+F9pMzzGhBq3a4uPV90n+aeBkUY3jmQyuaLN9aU916KIrnO7LffMbevj3Hbb\n/YtV9qk+rOtDPSOPfPXL0Hj/B39FztnbCwDojbrjtmqyxeNOxG0Jvs8WneOhyfB823rulvuwqo03\n3PVQY7LcibWy/sC/NZBxBEGCu1j5DoivW6Dv48pnNJGgZy2ZdPddnTM+XK0ZHvhffeKPV/R1qbga\nUhTW2q8C+OrV9OHh4eHh8frgql7orwta9JUOi/NxU9hF0mHXwI64zbAME0R0fLst0mSCv6a2KV/J\nF555HgCwuCD93nH3XQCAHbtJG7Dq2x2YdMd5qOOVX+n1/PZX+5rDHb/Ol95Jtxpt6K/5xWMFgoQQ\nzhFL9/m8aDERS6wJEaCR7yZpybJgkoFI7+k8XYdISWVOkEoGSkJiqcNpBV3dsi+ZoN+2E2r8LIEl\nUjI2d/8Mj7unVwbZbNK/9brSWPgWhQ1py2ZJMk+yBtJoLcnxqSaPTTicqL2etZHnrG5ZPAN9L9xm\ne6VUK/dbSXHSiTqMpUizsnt3/oTR0nXA/0pbEHTORa9RJ+FaLV2vsp7sMsm8vYr0rhHvX2dpfvrh\nP4233/XhjwAA9t30jrgtFUvESjJepjzrp8ZdK/2cOcm4Q0JPsLYWmBV9xH8Y/cy5ayRHttudz60+\nZ+A0J3XZV3tEY8k84brSr9uVz77FyvfA5cKH/nt4eHhsEfgXuoeHh8cWwYabXCyrR42WEFYLUxSv\n1IrE7JDuGgIAZBI05IRSmSPWy1NGVPAMay9nT4k/fnmRyLFb3vYmAMCu3cMykJB+kE6Jup/t6qK+\nNAEbdBItneofj8lqNdt2HA8AWsmiPpV6y6qeJmQdERergavA1srxdjJN1y3QKiSbNaKwps7F5i4e\ndjLXJcdbbhRtGIs1un5G9evMNIGbZ1PuWQt0X2qhtEURk4BWKcKs/iZZ37aBXKFWSPe0Wpf1gQT1\nF9aaMueAxhHxOSu1xXhfNsekYVom051nck4uRwynSnfeW7ex8ni70roimv1qP1jNTrHOOllmUKF9\nHSaUZf12WIVsx790/OtXR3g9v4FvfeWz8XbXCJGitx65NW5LJzPchzZ9cofxP3qd8FwCec7jw5UJ\nJcGmjiQ/j0HHPWATjbqoro9OMrnN+0znuAAEwUozTMgku35Gs0k2L676lnUml0i1eJOLh4eHhwdj\nwyV0RzDc+KY3xm39g+Re+PLLJ+M2x3+F7E6XUp+9NkvyEVYSHZH6OM/MzAEAvvvE9wAAb37rG+J9\n24YGAAAZdp0DgGyNiLVcPhe3ZXjbSc2akMpknHSvP/8sFWo3KSZ0W80Wj1QG6dzRWi05fn6eiN2l\n8iripEMkkkEySPNpRJKx7P6XK4i20WJCOsUEZaQk74gl9EatHrfVqy0ev5JMmNQZ6iOJN2Fl7q2w\nxfNUUsgqbFczIqk64vEmUuLaVmPtq95Skg+fM5tUDC9rbJETlxVxm0imeX4ytrCl3UI7YYNVxPFV\nCDbrJMZYOlQa0aodr+w2uZx0W4W8DJUGl0jwulMipiPzYo1MEW3twI1Rqw9OOlxtiDbeig9fRYIV\nyXVlH/HcKkJMv/zsUwCAiQ9ciNt27DrIw5X14bRcWScrtaSgw+3TSdxKyo9JTvq7rdakO24VL8MO\nbcOdwz3ejmil/rnfUN3l0Lk+CmLKlx0H7Crys26zdp2LeYnwErqHh4fHFoF/oXt4eHhsEWy4yaXF\nakttSfyo/8W/+hAA4K1vuzNu+8IXvwYAWGJiDpqoXMXVO2TlJ1KqtWV1rs2a4IlXXpPjD9NxwztU\nhkoX1dgSs0ODzTBOhU0lxXe7kWZzjfI3TbI5I2wJgdds0FxrVTKhNDWRyKaWxbKQnHOz8/y7dUjR\nlPhYt1PsU68i05oNOlcqkONMmsYZsaNspKMUWTWNIiEjC9kUj1FUzSabu6ptml/SKDMIm8WitpiK\nnImh1RB1vOHIbzZPNZSpqNZgc4xW93nNJKxavkz6Npv027Za2s0Wq/FldQ+W6J4O9u7EcphlxJyG\nXeWv1fzLY9JLdeLMJTbS5jQyAy7MTwMAKkvFeF/I6y6ZlBvT20vpp4cGhdDv7u6n/pO01kygmOxV\noqLXI0UvmTA1KzZWIB/JOKbOkaPDKyeOxW2DO/YBAKJQzhnGEZqud339nF+3JngjPl77ofO/ZqV5\nyqziPy9mI93I98qZgLQ5jTfDlib7V0aaK2aXz63iWViWDpUpsVRax6R6ifASuoeHh8cWwcZL6Pw5\nPT16Lm77yue/AAB44Ld+M267i6M8n3jiuwCAwLbifW2WwjV51GJiMK2+zs06SWjtJEkt8/MiBbde\nHeUtOX7nLpbWVa6JMKKvqLV0zqYiZ+PMKat8iR0BCohE0mKpvVQSqWxpiSTiliJc6jzuULpYgfSg\naBaRdWSn/CCo07hDRRYiRdt1lmoD5S7YbNHxRpNBLAWl00Ich41Od0/FzaLBrFSxJjllelizSqhr\nlGaNJtZm6jJuJ1ElE3IPkm06LpcQbaPGUcINXgtJldenFtL1s8oHMxGsLcusGvG7Gpb5K5qOyOP2\nirbi/AIA4Mxrp+K2iXEiCdttlsYTiiBEi/uSU06N0R8nOwhKun77jtwMADjyhrfG+5Imp4fYOfzL\ndF/suC7WRWiufXw2EG1tap7W+NFXRUK/+6feR30ktEYR+4wCACIdRc3X2yhVMplwrOXKiFx3v5UC\nLMeote6WgnYbdBGccRSzWi9O4m+pyPRG1T1Dcly2wM8ak7Jt9W6p8nM+PyPvoPPnZlYO9DLhJXQP\nDw+PLQL/Qvfw8PDYIthwk4th80M7KWrXc89TFt4vfv6xuO0XPvgBAEBpgVS3F59/Kd4XsB+rVWaK\nmMRQkZ/u89Vg32adgCgqUh+joxJZms3SmHr7emW8cBFhpGKlUioRkkvSpPxq3XaoiB/nl11aLAEA\nKg2V1jXk66CJIiZONHm0HBVFVLaZoOxTZodMjny7Na+a4bS1jSpd06At4wjbnPBMqbxpJlF16lZb\nYXMQq5zpgphjnNmopX3T+b5kFYnbn6axlZkkjqyQl6kE9aevnzNLNdtimsly4i1HOGeUqShi9boV\nCoGdTcj2Wlg1AZb2i3ZkfGx6WXmtXnju6bjtyX/+JgCgt0cicrdzSudshtq0j7WLVwiUOQEtml9U\nmYubyot0/0Yb1KZuAfYcuIMHKfM1q9lfLhfrhc4yMspW1OD7cuakOCKUS0SMJ7sk7iDJhLFLh9xS\n0d8BmzezKomc5VBfo5NcBewwYVf6yjszU7MmThj1Om33cBppAEjw+yjmPVVAi4sUDVU8RnmJ5qKv\nRibf1TGXpYqYVounyOx25uXjcdt4USWUu0J4Cd3Dw8Nji+CiErox5q8BvB/AtLX2Fm4bAPB3APYD\nGAXwIWvtwpUMoFmjL2xbRVKmc/SF/fa3vxe3uSjMu+/+SQBAVRFtzz31HE1GfYrbdmVuBUfmxel2\nFYPXYiJxsSRfyXkmcpJJLd0QXHRbIqXyjnDkpSNMASFa2upctQpJmOUKzb2hCkY48lS7MzkpP7l6\nUggaVyBimcuJ0lbSTSLL0aNKkq+x+OGk5nxK+ncEmI7yTLE7ZH1R3DgTLpLTcFrjpmhaLZZwswkl\ntdepv5qRPlKWNAVHGnVEz7HEVq/L8c6VUuR4VVgjpOMi5dLo+PN6TST6ZGZlgZR4nzuPkmTbLppV\npxNm0s0VMkgkZF/E0vX3fyAS+ktHiRC8+eZDcdvAAGl/ht09k4qsc8Ruq6nz73D0ck5JrhwBW2+x\npP6anLO7l9wb+3cckbFxbp1EW7v0uhxFK/MRuVV/ufVeAtVHhtNBj519JW6bnR0FAGwrvClucx6r\nWZ57O1i55iP1rvjqlz/P45b1cd+/pFS9AZPFHZQra246Unh+boLOFfXFbSnW7As9FEGuSdGI3ykJ\n9bwkOU9QbUneS4tVdvMdnwIAVF+S99n48VcBAOdn5LXZfcvbaEN40svGpUjofwPg3mVtHwfwuLX2\nCIDH+W8PDw8Pjw3ERSV0a+2Txpj9y5rvA3APbz8C4AkAv3clAyhzlsCMsm+CJcuastV+9WuP0wZL\nQ+/7+XfH++pV+jqXivK16++jr21pTtoaLOXVWDJOqXJsrjDCUkW+9GWWpPMqq18iDkzgzIB1FazS\nchKgztrGErcKlnHFJlyZtDDU7oJ0fE1J6LFEFazif+V+15B9TlIMMyp3Cdth28p2XOcEOSFLHI5b\nAIAUH6eT/ofspmXrMt7dXFqsVibJpDov0mT/EOXdKVZF62lxVk1dLrDCtvM23+9IlcILWRrXLnZ1\n1lhSymJZ5QCkBruCmozIZRHfo6imAriSay/9hRJlapwvluK2RQ58ayj3U8MZHl1wWXeXzCmXpmt0\n4YLkLqlV6RqFdblGjQqt/4CzYCaN1jvonFPnJ+OWZpWOP3JwRA5jN06XXTCsyhouzdP5+3bsj9ss\ncxqmI+9nZ+m3Djt7bNdfmbWwI23hMpikaEGFkMZdLcv1eO7okwCAe/e8OW5rsBpQcTyU5sXYb7ca\nihT8ve98AwBw9oxUv7yN80KN7KCgsX/42tfifYsV+u3P3isyquUAvOKCiMaLiyRVd/VQW1NrzFwR\nprtbuJAsa89zC/K+mShx0NgrLwIAtp08Kn2wZj9yo+ST6r7hRgDA6Rlx4b5cXKkNfdhaO8HbkwCG\n1zvYw8PDw+NHj6smRS2JTmvS5saYB40xzxhjnqlWq2sd5uHh4eFxlbhSt8UpY8yItXbCGDMCYHqt\nA621DwN4GAB27ty54sXviJaEqqGZcFGYdfkAOPXtycefAADsGRYC4733vB0AoLhF9A8QmbEwLzVF\nxydIqbgwQerUuXOi2tQ4NW1L5VyZY4I039sftwVxbgyu9K5m5PLGuEr11B+ZDHRtCpdnxFloQuVy\n5WpBaLNDixvr7bVNLpooavPxjaSoyHU2+dRb8g1frHIEJRN+RuUMYa/FjrwtLg1toybujdl+Mrmk\n2ZVsfkau9+AwFSVZXJJiEy49sVVMVYPvt3GmAOVy6EjRTEaWaqPB6XDV2BpsDmo5MrQp16/A5qMw\nrYjSdSJFj54aBQAUS6LaOw4tUu5r7TiXC+3MJGU82YCuaUvVPR0e5Jwrai3UymTWSYPmlEqptL6W\nrnMhLf0WWLWv1cSMVavSNerrozWfVGTn3ORpAMDI/lvitnSa17Myp8Wel3GNTp0AeGUFj/jwjuM6\nkSqISSIzNwsAaEHMMMdeobHdebtESAY5IsjrTG6n1BizTMondS4XXtfTFybitq9+4VEAwE//1E8D\nAB79P5+K9508S898aVGu38+8h8wv2wflOe8pcArvYycAAGVlxkoxoe7cfgFgxzb67dikvFPKbOrr\n76W+xrbvjff17af3V6pPrscNt5CJ6DvfwRXjSiX0LwG4n7fvB/DYOsd6eHh4eFwDXIrb4qdBBOiQ\nMeYCgD8E8CcAPmuMeQDAWQAfutIBWM6TYjuS1tO/unRUiyXW0iJJ7V/5v5+L991xO7k9HTx8OG6b\nqpJU2LTiMje0nb6iN73xBgBCkADAiWPk6H/0qJAr80xw9PYPxG2pFJOFTkJXAkqN+2uG8jV3ZF5L\nBcZYztHhKrhrVzhXgi5SGfmaTMTpIg+pZXExrk9Asvo1VR+O+NRaTHWJtBJOogiT1J3StdeSYC7H\nZd6U5D86TZLRnhQRoM5NDgBmOMNjIrfSvavalmvUU+CgoKbTGIQ0zGZpLlkloZeLtAY6stfx/XDu\ngkmV8bLPldZT4yiGa2e2KxU5wElXR3GbOmeIC/yBy8ophy+xltQzKNkcTx4nSXTx1dNxW/9PvAUA\nMMgFVmBlTaJN1zsdaIKc7kdJrd0ia1r92/jaG5lbrUra6NKiSMGDgyQddmb/c26LTPbblZK3zuUS\nl7ZbpyhDoq8n3u5tbAcA3HSHkJEH9lHQ09TY+bgt30Xugi5dUJAVKd8kabuotO5ahS76yLbdcVvA\n2uL0BGkFCSNS8L7dh7kPWdcXzo3TOdVxNS7sMjY2CgBoqjKKCdYUulSAWE+exjE1dSZu6+4jDXVg\n1x4AgN2p5sJxdQcPidQ+PDCEq8WleLl8ZI1d716j3cPDw8NjA+AjRT08PDy2CDY8l4uLDmwpFa9a\nJ1U2m1NEH5sRXFTlxNRsvO8LX6Dq4ne+TdKG3sz+nYt1Yd/aTGa0mfjcfehAvO/Q+98LALjlLVLb\n9BwTLV1dojo6Ysjl9tARjFPTpNYuzYjKW2Wf9yjUdQ07v6ORzjnLmx1FJJzJRRF9vZJexvUSb7kc\nKtqvvM3kVTotc3E3P86Bo9VsJqMKGcmF42o1tnNimplfJNV/F5vMtu/aFe87EY0BALIFIY9CJoxD\nNZdkstJx+lAVFEmzbUlfsmyaowgVERxG3C/ngVFBmyhysZBkQlTqaB2ffldUIdFRp5I7NPp3tO0i\nItsqJjGZIZ26f2h73NY3Ryp1WBb/9hQTazbOcSPXqtViM4gaa7abzCUVKyaDMhNxLu9IWhW4aDGB\nPTslZF3/4F7uVx7/gK+l+PuvLNaxGsw6fujbb7gh3u4dIGLwtnveFbcNdpM5qlYT3+0LZyjXy/Q5\nMkttPyJ9VA2ZOlpNIeVdsYuuPnGScO+IsxPkv7/rwH4ZL5PK+bys69dOUgTvUkny41Q5NuLkaU51\nrKPQ+V51dcm9Gj1Jz+hZdzyAg0fIv/6mG6iQx/ZeyRWT5drIO3rEJGdDHdN6ZfASuoeHh8cWwYZL\n6LGboCKxYsFLSXEJlsoiwwUgckIwpNlF7J+/IbkSJsaIDLrlJvnCtyP6zSsc9WeyQph2s+Szd8+e\nuG3nbidtKuKHMxm2XdRaVcipAkeb9neLFHx+jKTUs+enZH4u9wdLUoq7RMjkS1O5uzkJPQpF4u5d\nluVOk1hVvm4qlQsSrAWEKgNKd54k1jl2KSsUFHnJ0pvOhVPnKNrZukhI21nyzzOxOz45Gu+rdbnC\nI3K8E3QTSrJrs3SdZ+k0mRb3McP3W0fCZjnXTy0St9YaS/51vlaBYo3nmnRcYGUcXUprWA5X/MC0\nV7rpaQnISbOxUKvT+gUrzxOASVRFKo9PkhSZBo1tqE+kuDByhUpEmjQJ2p/O6ihnzhXCBHZ3Sta1\nc+ubvXAybusf2AEA6BsUIjHLLqmO7GxrstO5Murg0XjKa5Oiw4clf8xUgsZxdlbGMTFKLoHTk0KK\nFk9SrpfZk5TrpOdmiaScYTUtr/LwLM6RVlypiFY8/X0iTffsJk0kpTNk8n1ZOCvRt/PztP51WT+X\n2bFYIu2hrW68y9JqlIbYqFAf4xfG4rbXXiNt4/bbbwMA7NwhrqP9PaStpZX/biNQD+wVwkvoHh4e\nHlsE/oXu4eHhsUWw4SaXCicqSltRK1Os0kTqe5NmfS/gSMD5ljj9jjBDODgkqtV3nyZ/8rHT43Hb\ncEhE1dfHRwEAH37g/nhfhiNRc1kZhyNacgVRgzOs/tbYNHL6hCSon+UkSumMHH/bLZRwp6tHSJsX\njxJx4tzKI+Xj6lLwNlUSqDgtb0ekqKpioI4BEDvyaz/+iAnSIKdS2bIJIunqJypf+QSbj0xCJYvi\nqNculWhsOyfB2j9CRQrmLgixlGATRCYp56wyQRmp9KVJp8JyDcoB5ffvChgUS0KCWw5jbarETS5V\nr4SgqrnHZgS5fsE6kaKO/NWV5GMoC4NzU7fByuPTXCdzUZnkIpcuWbWB/c9dAY9GUxXJyNJ61ZGz\n1QqZj7ryUiDk4L79AICpMSISc0kx+XVnuKBDWfzQTx59ivoviD/8TvaVHtlJ/yaT8hy42AhdWzeu\n77lOrYx2r5h05g35Z3/n778ctzVmaEyNmkQS97AJNuD02JMnfhjvK/I1Taj11GrQutfpo9M5nv95\nMulEDTHHDA5R7d1ARUWPTdDYSguyxjJpmr+zBLdVdKozi4bK9Nio0xwaKr6hwr7/x15+AQBw+0++\nU8bYTc9LpGubJtYm6i8VXkL38PDw2CLYcAnd5TEJVO6SgKt6N1UqUcfRuO+wjlqbKtNxAxkhoHpY\nEn3xqEjQI5xHIp+j/tNL8kU+9xqRNdku8QfsKZI02adK0HV1EbFa4DwV/QNCpBRniBCZGD8Rt52f\npLZb75Kv88ICkVevnqJyd1FLud+x9KsjRdu8rUuiLYfm47IuzE65MgYgiaMSiVSRDml7F0sjySWV\n4pdFr6wSSQssKTaVdLuXC0tsH6RzDoVSTmx0lgillCJ+nECSUcSWy9HRYIlnbkak/O5e2lfICwle\n5TS4YVtIUUfiucjFhEoTnMzSMq8uqRw70doEVCbt0vjK9bN87Y0W0VloE0VBSVsuHa1yHc1zv905\nkaCHBmheOSY5kyrfTILvo763eZZSdXrbfpb2gl37AQCViki8rvJcWpHELg1ztSzE4Mnj9CxUlsil\n8tChm+J96bR7rlYWvcA6kaKjM3K9i5ZTKU+KBFsucQGZQLnXxto5F5FQ4bdtdmc1KWnLubwqaVVq\nkodUXiAJOa1IaGPJfbKu0l739xMJn1cplZ22tVCmMTZCOadLoWzU+nDrqVyRdeW8exdn6dqGFfU+\n6+MSd7qQiL16+dpL6B4eHh5bBP6F7uHh4bFFsOEml4AdQ7UJxazi2+rUX6fBdh5P6suCqgY+MUuk\nyuKSqHhtDh/c3UVEVPmUmGMuZKmPob2iarbYBFBdlEomuSypeL1MxA6ohDozfeTf+63vPhu3VWts\nQjGiEk5yWt4ME4rzZSHJ4iRTKouW+PquzUDp6ktOjWuqWpRoO2JVVMLd3O0h/jejzBRzHKmaUL76\neT6urkwLw/2k7odsS5lbUqVljYtYlXNmOfI0q1RkN7+ATQw6KdZ8qcTzU6mAG9RvXfnlZzni0vAC\n0WmQk47gUraLhZJUYF+OIZ5TtSrmqSUmNK1OIRuvXTbH6BTJvF4HBrfJuPfsp7HV5Rpl3fVl0aqt\n4gkcidthLomDelfWrR0aorVot8majFxqZPVIZZj4T6kI5SaT1NPsMNCVF5PY3n038Hm0/OeqHq1t\ncikp3/B8D5kmB4b2x20D2yiKtrdbCN7pMapoVJyhKO2mVeaYNvXXlZBxZHi7VZdndGmJsnmHLuFY\ntwqr5viKsTNn4yb3nBTycp0LXTT/Iq+/UJlGXPyBUfdA/Bqkj6hN93Z2nkxF33ziuXjfkZvIvz5Q\nzxwCHynq4eHh4cHYcAndSQ6rSeU6Us+5klmXqlQfxxW/A/XlTnHqSswI8dlkSea1KfqCz39L8mG8\niatv33KrtO04QtJ6uUsiF7OczrPE+TgKRSFiu7r5uEDa2lyl/fSoSARHT5Gb1DvecQ8AYGFWSKyY\nFFIuirfccBAAUFySHCCAir5EZ6SoS/WZUNcj4K9/XkkVhzgXxe07nUSqUvAuctEG5caZZAl6URGK\n6Rz1W2R3u3JHOhiXP0akfEdiLSoJ2Ul5GZaQUhk5vsY1Z3UkYCrNEp3Kq+LWRYKJW339nBCZVBGU\njbDz+mmMbCcNbmJKJOmKI1QVCejGHTgXvg5plbZTKqI520OEXKDunWGJ0bno6jTLKZbyQxU1HLLU\nnlIV54PAEcIkzaaVW59J5VeObVldXPoNFwFh19TirBSMGN5B7ofpnLiTxqltgrUl9Ehpcs6tsNaU\ncfdyquXtuw7FbYPb6Zlrh0R4z80LcVucJek9rMgzHbVpvOWqtDnmvYcLfvSqiNihbbu4f1UjlCOO\nF6uSlndyhqJvmw3n8qq1Y7oHRq2xiN01kipXUrZAGshSna7RD184Fu+bKdGz1t09KN0qLf5K4SV0\nDw8Pjy2CSylwsQfA/wIVgrYAHrbW/rkxZgDA3wHYD2AUwIestQtr9bMWnG08uUoVdu025ipdmSRJ\nhG319XcVuQOV+TDH+VQSXeJGZxbZ+Z9dv6bLIim98CIFWTTKYrO7tUZf7oG9UmG9wvb3rj7KktYK\nlQTG9tDbbpesjzOcgfHZF6Tid4aDGgbZVjs5IRX8Fjl74Q0H98VtN95AUsU/PSHJ8/uHJHgJAJrK\nZtxim3Uw//KlAAAgAElEQVQuLbbJNEtIORV/dHgfSS63HCHb//kxCcIqnyUJWrtVVdO0XVaBUBkO\nqpmYoeNLDbkvzl0wqeyELb6nDRWc1MUuiY22c89Urmp8v0NViqzNrpeuWAYAJNg1Lc8lzBqqyEjE\nkmioXOC01rAcOe7DQJaz2M5XkUjXia5pKymuzkF0xmrpkPfzmtDPgXNhtUoTyWdXujJalymRD2ur\nQCTHT3S6vLJGoTpxdmGn1TUaKmiGeYlMXt2DVea6HJHKmhmyK/LITinowOlSUG3oXCsc4MdFT/b0\niFawazdJ8q2mcE4znEWyce7VuC2VpOu2dx89Q729isdYYolePec93XS/a6qATMguzu59k1D3LJWk\n7ZxyMe3up/dBT588t0cOvpHPT9aCvm2SiXTHThrT8Igcn+GAxG9JapvLxqVI6CGA37XW3gzgLgAf\nM8bcDODjAB631h4B8Dj/7eHh4eGxQbjoC91aO2Gt/SFvlwEcA7ALwH0AHuHDHgHwiz+qQXp4eHh4\nXByXRYoaY/YDuBXAUwCGrbWOOZkEmWQuG06t7FCBjSN5VK1Nl8LT/a0I09BFP6rj8+zGNtAnLlxj\nC0Qqtlgl1X1YJp7CU6LvVBeIkLuD87EAwNAhUpWKA2RKae+WOqZtVg97esTM091DquOLR1+K2/q7\nSZ0rcChZsyWEX38fjfudb785bjtxkgjVyUnJx7Hc5FKt6KhG2q41JJIyZBc7G4g72kvjdD36kqR+\njvSKiaaL3bbGFoSwvcB5M2qq6EWe+5ti8rLcUCl+uyh/zWJFRXSyuUaiD4GAiUEX/RgqVzXLZG5d\nRdFFHOWX0ulcmQiznKpXm1eaTICFyvwRmPWW/rK0uBD32rbOveHc4uL0strUwWSajiZkIjZQUYdt\n2+lSqU0uKTbDWJUfxLrfqlwkaTYLGDZ/WTX3KM6To6J1VyEyo8jVR3XmGFWQZVW3WaP+vzraal07\nHj2t3E/Z8hkToABgeF4trqdqlKktYDOITckaHt5D5rq+IXn9FLL0m+HtZHbVFrHxC/QMDWyTazQ7\nSwRoI5T1f/gGev4iJvtrFSFMKxU6ftewnPPGm38CAHD3PSLXJhO0xuc4UrS7V/I59Q3Q2Hp65L2X\nSF3DSFFjTBeAzwP4HWvtot5nyQC3qiHRGPOgMeYZY8wz1Wp1tUM8PDw8PF4HXJKEboxJgV7mn7LW\nPsrNU8aYEWvthDFmBMD0ar+11j4M4GEA2Llz54qXfiLhijyoPCKODFISlUv05sgbXaKtFedCke+T\nTVLbkMpeN8llp5zrV6QyGi4x65pQkuCJacopsvRtCQi4fXY/AGDfzUQkLixI3pE6ExyFYSE/8r3k\nyvjT77wnbitzsELSkgTzhsNCjBw8wBXC1ffxmeeIUG3W165UX1kUoiidcFKZygvChMt8Wfp4lZP3\nH8zSGHf0iMtVgyXYBRXgNMO/VfwrbJsk86lFcs8MlUTaaLhiICpYhoNZmsp9sskkborJyGxGxl1j\nIaCiAsQyzmVPubrWG0TANV35QhUQ5c6pFDKE4dq0npOWEyoHiBNwE9q9lrcNVz+wHbk4VpKoLutk\nu6VzhtC/yYQLUpJBujaoXDgRS71BoAJSuPSdc03tcAF2zw1UIBJnq2w0ZP3b2B2SXUeT0r9zaTSr\nuGyuB61ZRE12wYQmplkrSSgimK9hOkVabk9BpNpUktZHIi/aaW8XXdN8WmlObVoLk5OUn+nY8Vfi\nfe02SeHbhsWVMZ2ncwyod1BPD5cO5HWyMC8l/KYmSWMu54T0haGxnTotBS5ccGCFXZz7FMHbz3mi\nSiVxRc4VxMHiSnFRCd3Q6vgkgGPW2j9Vu74EwOWfvR/AY1c9Gg8PDw+PK8alSOh3A/jXAF4yxjzP\nbf8JwJ8A+Kwx5gEAZwF86EczRA8PDw+PS8FFX+jW2m9jbe7j3Vc7AKeWaJ9zdzqrChKkM6T29Q1w\n7hBVT3Jhho5vKAIFLrJQqX03c8TliyepwETY4ctLqm5FmQJajqRT5GLj5VHeSWrrvgOiIp8/8x0A\nwJDyIR++gSLfhoZExTuwm3xWHZF3+KDkmshwGtUnv/3duO3cBVLxwqqydSxDXuVysRw9V29qlTDW\nvQVsWgjZNFMPhOwc4xw0J0tiyilxbpOUFVKqyedyJT+dCQ0AqhVSKwsZMeVE7FNdVWOrc3GOXIJ9\nkFVOjSYToIEy/bhK75o0Tzpi0NVfVdGVSDrCUeWPWSftq+urUBCzTZE1Y2c6AADn7u3MD5Guk7qi\n4AaQ5ajNclmiGluuBipf+5QqpOA4skBF60bMBOsCHZa3AzbpJJQfepsJ5raV61Gt0DlHz0id224m\n57r76Pka3C7qfy5D47YdkdvuHGtfR2PkHieTHDUMaetmf+6UMg0aNg32DNC/Q0Pb430DA7TtfNQB\noFwisnLsvNQqffH57wMAXn75aQCdeX3uuutnaS563Pxs9KicLwk2OaYydG17e8XRIcXvovmymEvG\nxsnivFiRWJEmv3uaTXoOcmm5po787lFmTpeaO3cV3KiPFPXw8PDYItjwXC7u66kj2Zy0nlcZ35JM\nhPVzZrZdI0KMtI6Q9HF+XCTp4lEiJ8pdIu0NsSujy/RXqynShs8fKre0Jn/Fm+pj3uQK8i9ytfZd\nN4rk3Z4ggvT4sR/EbeWT5Nk5dESO6zlE2eu6dpEkv2eXSCFnzhL58oOnpI9Wg6TkckVJ3MsQKAI5\nyRJava6IR85MmE2oCvIsGS0xT3V+Qa7f6QU65+miSOgtlpYH0ir6lkV+J2VHgXINZMk4q6rWO7e/\npiKgmhyV2Ob7kcupCFeWOluaOHPl9JSUmuKovQKXZotUJGrIpLnOUGiitSXLNLvO9Sr30yK7vNZq\ncg/aLLG6f60i/AxrD1rLTLIWFSgtxuWcqVboOqdUlqJkF0ve2lWSr2lH6iN2V2yxu6KF9O8kUZ3V\nb5GLNpw4/lrctmc/kfwFLvYwsF2iow3/1raV/BfnYMKaWCiKw0CB575dedtuT9H90ERzoZ8OSPbQ\ns9+/TaTmCHTfjx+XLKnjnJ1xdk6inF8+RS7CE1xwprcgfTRatMZrC5oU5/uSEUne8dFJvleNpmRz\nDJ1W2tJrjO7fT97xlrjtwhhJ6+fHaV8jkjxRNS6E4UrXAcDCNJ3rhsOSO+py4SV0Dw8Pjy0C/0L3\n8PDw2CLYcJOLM6+ESkV2xFpeVTbv7WVdjbXJ2qKoaQf3kEq1a5sk4TnKEYzjC2IymKuQqrTvwAEA\nwHBTEhBNTJEJZUkFPzl1q6VSty6yCn1ikYitQ0dH4317kqSiLzVEvT3xPaovOnFKVML8XiJweg+R\nyWXioESFtvI0l6ilkiOxOh6t8/3VFcv3jFBKzqSqhbpYI3WyXFbFOjh9bpkjKV86eyHed3ycK7JD\nRbLlabnUFBmZ4e04U68aYjZP/VdU0jSXZ6quikc02YTi0u02asrU5pKxKZNcyOYdo9IDtznNaY4J\nq261dpwRY6mqzSVYE4ZTT+kiDwP9dC1LJVGRa7xmW9xZLiWmjh4m7uoqcrZWo+vR1ycqdSHnIkpp\n3enguySTrFk1Dpe+t6NgRcaRoRxxq8xTjkBOJFWStfoMt0m/3b3kIz3MZsCuflUkwxGEOpo7Jn3X\nTkx2eM8b4u1BjpheOi8pZPNMUqd6xNd8aCc5DBQ5ydtzz4npcXaBTEVl5bvdark0z3Ltd+6hBGAL\ns/TM1epyTccuUCT4gHJSyHDcQ6RSKoctF9fgosplXkkmsFs1MaHMz5zgc0pEeKtKz9riPI2joR6O\nkBOBJZR5LB3HFniTi4eHh8ePPa4bCV27Lbqv7pKODuTCCFmWwPKLKpfLFFfoTos03ssSUv6wSO2n\nzxFBWZqjL70mfoZ2UoRmU1Vpr1ToCzyvcqgszJGb1CyXvDo9Ia5fhV76Pm4flBwPNc69MTspkl2e\nXfAWlmi8zx0Tcurgmyn17p133Ba3tZ6lr/7UjOSTWI4uVcbLpedNGlUogiNrq8rdc4nTA0cJIsSK\nSjp0LqA62jSVZ2lZ5Zxw5F/KRVUqgc2VNYNyJw2Z2NUameVovFSG/m01RPIpL9FxdeV6lmNJMVSE\nY401GkcI61wuzkMxkxHNKZldO32uC9Z0OVIAYGiApKZSUaTDcS6K4gjYPuXadtNh0gLnSzKXKkf6\nlmqiTRUKNCZX+KNZlTU8P0drpnxe1ljIkvHefZKGtjpJLnPOBW5wQEVXxtOUe3bLm26lDStudJks\nR6py/pGWIo2zTPqeOXs6buvjEoyDqgTjcrzn5+6LtxMRzb22R56NrKG2M0q7/NYPKSrblR6sqjS+\nLY74jZQrqIEjLdWJWaPoZy2jND+j9rmIWLkeNX4O2m3pxCxLiZxQWs/Onftp3K/JMz3KmsdnPv+X\ncduObXRckKB1kVJaYzpOLa2K0ODq4SV0Dw8Pjy0C/0L38PDw2CLYcJOLq67ToYKzKl0qaTMMqV5p\nVu2DXiH8xs8QmXd+VvKD1ZgM6totZpWRvWRWcZFpJaUOB0wsFQriKJtiVbN7QMbR3dXDvyWVMBgW\nlfPwPe8FAOzZLn7lBxbJF/el5yXB10lO11nNcZV7lWjp1OlRAMC5SVFDjxwhv/WhwbVNLv1Z5Zdf\n40hRTRqy3UGbZvLs2z9eZLJJkZeZApsA6oooYmIooarF1/k3CeeTrcwUTdviY1Q1Ja4Yo1zIkeY/\njKtepYqft5k8bVSkjwyr1DoiMs0xBin+sT6nq7GazknK3uTaFhek2We6raJJe3uobe8eWU/FRa5F\nWa6umFOBk0alMmL+mJ0js838jCR6qnIUco8LD1QJwaZnuB7tWamr6VLk1lRwhCNUR3bQ9c5k5AJ2\nu5SzynQ2PUf9TS+IKaKLCeyZ778IAGg1hTDd3k9r/pMPPxy33X7n2wEAv/RLa2f8yPXJWqty8riJ\nqpgpnv7eN2h+E+IwIDVTaQ5dBYlhaEau8pROBcz3VFlIXDzA0BCZW8OGmLFybOJKqwRw8/NkPg1b\nKpEaV+fK5ug6RCqZW3eeTCj9w3vitrFJMpvOK5PZ3j56Jguc/C6hEqS5GAodWV0sdSSxvSJ4Cd3D\nw8Nji2DDJfQeTmk7uyAkVugKiKq8D05CP8f+Wrt3SIraG3/unQCA5uho3HZ2lKK0xhShmeE8HNv2\n02/TSeXWxxJdRX1hI5bs2qogQJ6Jr77t5BrYVpGXcwU67i233xS33TxChOO2n7o9blv4PCWmPMNj\nVGU+cXaWpIVaTYgw5/p25PCNWBM1kUjbCY7GVJUfKyzpZvQnnCW5CZbQI1X0IUiTZKLd3RxhbDVh\nxlpAjtm3iq6E7iQqVdDB+Tc2lcuok8IqnPslrWuFOkJTpXNtc9rXRFom0+L5J5x0rSSqmjs+I1Jn\nW0lLy1Es0pqJdAEUln2ayp00wRGcEfe/qDS+CY4kTqi1YyznwlEEm9NQIy7o0FTOAUtO+0nLuG96\nA2lr3d3SlnDRuZyvpaX6cJGiVsluCyW6zm2Vz8SlyqmwNPv0D74vY2TScGFeaqyeOUMS6fiEaBvL\n8eS3Ho23X3qJNFSdx6aXtcVcr4ok5mjegN8BQULWjhOqVYnVOJeLSwlMfbBGFtBvsznpv15nR4d5\n0ebzOU5ra9W95WcoyedPqufApSc+ePBQ3Da0g7SYVErWXVcPk84cpd2RTjhy5L2ui6vzWV0ZvITu\n4eHhsUWw4RL6G28ke/Nr4yJxlLmQglHZ65x9ssY23SdUrpOz4xSMcNMR+WLeeBeVhJpWdvW5aZIw\nXFmwVFq+3Dm2TeYTYlx1GeKqFSVBsO2yv5+ki6EdcnwrQ25df/8NkULe+Ia7AQBve8fb47YHfpOk\n9i9/8YsAgP/39a/H+5Y4+KmnS1zKpqdoDnPKbfGd7/0ZaKRVtkUXddKqikQQNlz+E5HKajznrDPf\nqvJnDXZzbCtpvMUaTlaVqosi1mxY4jZqHEk2bPYqO2jIAUD1tiqTxlJvXLhCwUmY2YxcZ5eTxaq5\nuFwlrmaDSn8Cw9JesynXI2nWDoiZY9fUTgmd8/qo/B0F5kB27yQbaU25W556bRQA0K2CgvhyY7vK\nILg4T9Jv6M6lcq7c+CbKC1KpymScRNrdK8En24dI0xwfJ2lZ2/4j3s5l5Z7t3U8ula40HwAc2Etu\nkO7+qVQ7ePUFKhAhhWSAPs7K2GyJRrscLygJ3ZmnEyr4yvDz192lsi2yjJlg11tdLS9I0R/Vmqyd\nconzAFm1dnkthrzGmsqnscx5bA4efGPcdugAac8vvfBM3DY1RcF/yTRdiEKX3MdBzvqoo426OVhM\n592xLVoPLpOrdoV0l14ptEimdNGSK4OX0D08PDy2CPwL3cPDw2OL4KImF2NMFsCTADJ8/OestX9o\njBkA8HcA9gMYBfAha+3CWv2shTfcSOrRzgOS6H2uSOaGsjJ1LLE2OztFKudSWVS9l46Sq9XYmCSX\nv+kGIo8yOaXi8WyrTBCmFUNY53wPgfKZ69lB4xgYFHWrNO2ILfp7oE/UpNIs7Tv+kqT3fP5ZUoPP\nn5dag7/wgXsBAB/9tx8FAAxuE9fHx774NQDApIpAHeScLDVFJC5HNqOIHy6aYHRNVt5uKvfQDKuw\nhl2zaspF0QbsdqdMAA1n9tKFNni3y3ujc64kORovp0jAJu9vKl06xWp+nsmrQJnCWuzutlSSHDTO\npS2TVqYfNi2ETUeIyRC7CuTaputZBusUZlhi18qGqjkL41LZyvpwS6W/l90ol1RkLruOlktC7Hfl\nyI2tq0tcGctF52ZJ6y+tTCNRnfTyqnInHdlFhP7AwKCci10f3XizGXHPdMRnST0vF9gldnjHzrit\nm92AIyZW2ypV7sgIuWqWDkqU7O7dNA4dWb0cwz3KVMkXq6b8C1ucyjatojaDOEkNryNtcuF9Cc2K\nxpl6lAsrnyviIiptFTVc4UjlYlFMsRmOUO5TpsS5Oeqvm913LfSzRHPOKxNlyAVVAuUqnMm62r4c\nDd9WhVt4erVQuWa3rw0p2gDwLmvtWwC8FcC9xpi7AHwcwOPW2iMAHue/PTw8PDw2CJdSgs4CcGxP\niv+zAO4DcA+3PwLgCQC/d7kDWJimr3S+R75O+0foyx4F8jWvlGmok30kNU/NiNQ8O01f1oYivc6c\no2CFXTuEgEqxROrIsWRavurOda5UVAQKb/d0y5e1t49L1S2SBHbyZTl+qUxt1UWRqAJDbY8+9oW4\n7ZVXjwIAfu3Xfw0A8OBHfyPed3D/YQDAp/7203Hb6TOkeSysE3gQGcUC8me6kJVrtMQ5MYyShrJc\n2qxSc/lVpItUnqRNzR2G7GIXqOCXgPNTpJm0rKt7YFkaTylxue4kQFVSLslRPpbJak1iuRw+UUYR\nYVz6q6kKVmQ4IMyVg0spTSvHeVtCJcW1w7WloUXObllt6OyM7MamsuOBCeFTJ+h+LsyKBDsyQkEn\nfdt3yLgdqawku/ISrf9Enu6PdgmdvEDjGN4pJQ37WFvTgS4VDnACz0lLaRUmEC+MiftuXz9J9729\noik0mJhOsfdBRgUi7eJgqoEhlSUyT9e7WV/b/bOiywA6QlUNLsGLK6GIdLfbrTurXDAjJtS1n2/I\nWonir+MMhnkmrTNJcTAockDgsVdeiNtKCxT8l1eajeV7W69RXylFWDbYRTaj1qR7VbWbcl9CnoR7\n7yTVI9rie9VSBG9DuTBeKS7Jhm6MSXCB6GkA/2itfQrAsLV2gg+ZBDC8xm8fNMY8Y4x5RqcG9fDw\n8PB4fXFJL3RrbWStfSuA3QDuNMbcsmy/RWf5Yb3vYWvtHdbaO3R+cw8PDw+P1xeX5YdurS0aY74J\n4F4AU8aYEWvthDFmBCS9XzbGxlktKQkh19NDukmvMsP0ddNQtw3RR+HAHhn61AwRqucnRN2fniR+\ndmZaeNodI5TbwaXO1H61TmXq6pFv3EA/++Qq4qdcou0WJ+CvqwhNl/ZXV1hvc+rYujIPfPv7VI38\nzHkyC/3qv/nleN/7P/A+AMBhTr8KAJ/4BKXkfPSxr2ItaPNUgkPqrDJrOBNEkNS+1XztOR1tJi8q\nnzM36FSlLnovq/LGuBwqbbg8G4o84rwcuoam8+NOq4+7I2pd+tJKQ0U6ciRuSsUkZBKdBR0AIOBz\nuRqnKV23k9XnbEFIy+Ls2tqiu26hWh/Ol7ihiiW88uLzAIDXjlPq1HpD1OfZeVLt7xociNscX1ZR\n0ZIZJtuipouqVWue079WFQk9MUa/1UTiUonMKZl0i/tQeVumyYkgmRRzwsgwmYGMWh8up0iwioyX\n4IFrgSwV5+Vdm1wuq2ejWOTnUJkGuzlSNJ8Tc4ZlYtAwyZ1SCXICNs3Vqur54usVKVNYm53o0xxm\nnMuLyaVtqT/tdDDaGAUA7Nwh5jE3q2qN+1LFPZy5MNEjJs2I10pdORZkOD9PgudcV/exxpHpumiN\nSa2TYOgScVEJ3RizzRjTx9s5AO8B8CqALwG4nw+7H8BjVz0aDw8PD48rxqVI6CMAHjHGJEAfgM9a\na79ijPkegM8aYx4AcBbA2mnX1sEsk4vhonxbClxebnhIhreNo756Ior0yhn5qo9wpFw6I8UschmS\nIi9ckExu58dJiSiw+5iWHHu6SFrYu0d9zVnafO2EkJFFLmnnIjONigxrsIQWqkx/rhr9bEWiCBu8\nv36K8mH8+V98It43Pk6ZIz/6Gw/EbX/0xw8BAPq3ibQXKekKAJot7WLHVe5VeTCXXa6hXOBqljSa\nwUEiybLKZa41S5KdJr0C7k9Xsq8z2RpZd04ZhiuppQnQiF3IApVhMs3kbCFNUmRSXb9qnY9XkmuB\nXcnayoXQaRuuPF1NlwyLXNk2GZx2OVuOFO+zoSJRWXJcmBVysThL0rLhPCJLNeVaWaY1UywV47Y+\nLkBRVNHLKWbKegdIOkymlMthmeZSLss9CzkislETbdSRyHkumjChpE+bpHMeOXJQxr1A9/bcmdG4\nbdcIlWQ7cHg/zSlYKevpsncun4ldp5afzhWDWJuS45tM7Oa0q6t7JmOWU+6ZW0flsmhJLoK3rV3+\nWBuuN9iVNdQR5yQFd6uyd27N6kcqyZGqLoLXqucrwde7OacITdZGtN05U3frjsaTUms+x1lbcyoC\nOnH1nOglebm8CODWVdrnALz76ofg4eHh4fF6wEeKenh4eGwRbHhyrjoTjmkV9ddkf9OZORUZGbEv\n8RL7hapC6Ok8qboj/XK8S5yUyYoZZnKCzCUVTlqlo0LB6VTLJaUCsYqkSamGq3daXZmUqM26W0qZ\nJNoRqctDqrhCK0dzdcUKdNKoxx77MgDggoos/dVfI3/1f/exj8Vtf/YX/63j3A3pIo44s4ooSoLM\nGnUVNVfnuRSLZA7qLqiCGEys1RZlnk3+bS4nF79V4QRIrJKajirwNI5yXZHPTP5Z6FSlAR9N+zJK\nznDd1evK55eTeBXyqvgBz8WZXOrqnJmQo1hVtF+fSqW7HOmEq/Mp92yRYwCqS3I92qE7J0enqmjd\n3fvIhKHv7YXzTNTPzMm52CQ3WazzORX5FpuZxHSRc/72yj7g4gfOniNTjlEE+b5DRK5HqijJy0ep\nRu2Jo8fittIBInG3bafnpXdQUku7exAom0tscsHamJiWZHLOTOdMkABg2RSRVRHbjlR3RWh0/01O\nQ+uSxAGA4VeYNgdZXketFj9f2hrpok1TOmUvHa8jpTNMgrrrpmseNznpVksFbrjdOsFdjs1jPb1c\nD7kg5tyuHCU30xHNgb16t24voXt4eHhsERhr1/vGvr7YuXOnffDBB6/Z+Tw8PDy2Ah566KFnrbV3\nXOw4L6F7eHh4bBH4F7qHh4fHFoF/oXt4eHhsEfgXuoeHh8cWwTUlRY0xMwAqAGYvdux1jiFs7jls\n9vEDm38Om338wOafw2Ya/z5r7baLHXRNX+gAYIx55lLY2usZm30Om338wOafw2YfP7D557DZx78a\nvMnFw8PDY4vAv9A9PDw8tgg24oX+8Aac8/XGZp/DZh8/sPnnsNnHD2z+OWz28a/ANbehe3h4eHj8\naOBNLh4eHh5bBNf0hW6MudcYc9wYc8oY8/Free4rgTFmjzHmm8aYV4wxLxtjfpvbB4wx/2iMOcn/\n9l+sr40EF/l+zhjzFf57s42/zxjzOWPMq8aYY8aYn9yEc/gPvIaOGmM+bYzJXs9zMMb8tTFm2hhz\nVLWtOV5jzO/zc33cGPPejRl1J9aYw3/hdfSiMeYLrhob77vu5nC5uGYvdK549AkA7wNwM4CPGGNu\nvlbnv0KEAH7XWnszgLsAfIzH/HEAj1trjwB4nP++nvHbAI6pvzfb+P8cwN9ba28C8BbQXDbNHIwx\nuwD8ewB3WGtvAZAA8GFc33P4G1DtYI1Vx8vPxIcBvJF/85f8vG80/gYr5/CPAG6x1r4ZwAkAvw9c\n13O4LFxLCf1OAKestaettU0AnwFw3zU8/2XDWjthrf0hb5dBL5JdoHE/woc9AuAXN2aEF4cxZjeA\nXwDwV6p5M42/F8A7AXwSAKy1TWttEZtoDowkgJwxJgkgD2Ac1/EcrLVPAphf1rzWeO8D8BlrbcNa\newbAKdDzvqFYbQ7W2n+wNq7i/n0Au3n7upzD5eJavtB3ATiv/r7AbZsCxpj9oFJ8TwEYttZO8K5J\nAMMbNKxLwZ8B+I/QxRw31/gPAJgB8D/ZbPRXxpgCNtEcrLVjAP4rgHMAJgCUrLX/gE00B8Za492s\nz/avA/gab2/WOXTAk6KXAGNMF4DPA/gda+2i3mfJTei6dBUyxrwfwLS19tm1jrmex89IArgNwH+3\n1t4KSh3RYZq43ufAtub7QB+nnQAKxphf0cdc73NYjs023uUwxvwByKT6qY0ey+uJa/lCHwOwR/29\nm9uuaxhjUqCX+aestY9y85QxZoT3jwCYXuv3G4y7AXzQGDMKMnG9yxjzt9g84wdIUrpgrX2K//4c\n6A/XF54AAAGHSURBVAW/mebwswDOWGtnrLUtAI8CeDs21xyAtce7qZ5tY8yvAng/gF+24re9qeaw\nFq7lC/1pAEeMMQeMMWkQAfGla3j+y4ahAoSfBHDMWvunateXANzP2/cDeOxaj+1SYK39fWvtbmvt\nftD1/oa19lewScYPANbaSQDnjTE3ctO7AbyCTTQHkKnlLmNMntfUu0F8zGaaA7D2eL8E4MPGmIwx\n5gCAIwB+sAHjuyiMMfeCTJAftLajiOemmcO6sNZes/8A/DyIWX4NwB9cy3Nf4XjfAVIrXwTwPP/3\n8wAGQSz/SQD/BGBgo8d6CXO5B8BXeHtTjR/AWwE8w/fhiwD6N+EcHgLwKoCjAP43gMz1PAcAnwbZ\n+1sgLemB9cYL4A/4uT4O4H0bPf515nAKZCt3z/P/uJ7ncLn/+UhRDw8Pjy0CT4p6eHh4bBH4F7qH\nh4fHFoF/oXt4eHhsEfgXuoeHh8cWgX+he3h4eGwR+Be6h4eHxxaBf6F7eHh4bBH4F7qHh4fHFsH/\nB2S3zfjmVWyoAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x29560ef8898>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "plane  deer   dog plane\n"
     ]
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "# functions to show an image\n",
    "def imshow(img):\n",
    "    img = img / 2 + 0.5     # unnormalize\n",
    "    npimg = img.numpy()\n",
    "    plt.imshow(np.transpose(npimg, (1, 2, 0)))\n",
    "    plt.show()\n",
    "\n",
    "# get some random training images\n",
    "dataiter = iter(trainloader)\n",
    "images, labels = dataiter.next()\n",
    "\n",
    "# show images\n",
    "imshow(torchvision.utils.make_grid(images))\n",
    "# print labels\n",
    "print(' '.join('%5s' % classes[labels[j]] for j in range(4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. 定义一个卷积神经网络"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从之前的神经网络一节复制神经网络代码,并修改为接受3通道图像取代之前的接受单通道图像。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "\n",
    "class Net(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(Net, self).__init__()\n",
    "        self.conv1 = nn.Conv2d(3, 6, 5)\n",
    "        self.pool = nn.MaxPool2d(2, 2)\n",
    "        self.conv2 = nn.Conv2d(6, 16, 5)\n",
    "        self.fc1 = nn.Linear(16 * 5 * 5, 120)\n",
    "        self.fc2 = nn.Linear(120, 84)\n",
    "        self.fc3 = nn.Linear(84, 10)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.pool(F.relu(self.conv1(x)))\n",
    "        x = self.pool(F.relu(self.conv2(x)))\n",
    "        x = x.view(-1, 16 * 5 * 5)\n",
    "        x = F.relu(self.fc1(x))\n",
    "        x = F.relu(self.fc2(x))\n",
    "        x = self.fc3(x)\n",
    "        return x\n",
    "\n",
    "net = Net()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. 定义损失函数和优化器"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们使用交叉熵作为损失函数,使用带动量的随机梯度下降。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import torch.optim as optim\n",
    "criterion = nn.CrossEntropyLoss()\n",
    "optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. 训练网络"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这是开始有趣的时刻，我们只需在数据迭代器上循环,把数据输入给网络,并优化。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1,  2000] loss: 2.286\n",
      "[1,  4000] loss: 1.921\n",
      "[1,  6000] loss: 1.709\n",
      "[1,  8000] loss: 1.618\n",
      "[1, 10000] loss: 1.548\n",
      "[1, 12000] loss: 1.496\n",
      "[2,  2000] loss: 1.435\n",
      "[2,  4000] loss: 1.409\n",
      "[2,  6000] loss: 1.373\n",
      "[2,  8000] loss: 1.348\n",
      "[2, 10000] loss: 1.326\n",
      "[2, 12000] loss: 1.313\n",
      "Finished Training\n"
     ]
    }
   ],
   "source": [
    "for epoch in range(2):  # loop over the dataset multiple times\n",
    "\n",
    "    running_loss = 0.0\n",
    "    for i, data in enumerate(trainloader, 0):\n",
    "        # get the inputs\n",
    "        inputs, labels = data\n",
    "\n",
    "        # zero the parameter gradients\n",
    "        optimizer.zero_grad()\n",
    "\n",
    "        # forward + backward + optimize\n",
    "        outputs = net(inputs)\n",
    "        loss = criterion(outputs, labels)\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "\n",
    "        # print statistics\n",
    "        running_loss += loss.item()\n",
    "        if i % 2000 == 1999:    # print every 2000 mini-batches\n",
    "            print('[%d, %5d] loss: %.3f' %\n",
    "                  (epoch + 1, i + 1, running_loss / 2000))\n",
    "            running_loss = 0.0\n",
    "\n",
    "print('Finished Training')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5. 在测试集上测试网络"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们在整个训练集上训练了两次网络,但是我们还需要检查网络是否从数据集中学习到东西。\n",
    "\n",
    "我们通过预测神经网络输出的类别标签并根据实际情况进行检测，如果预测正确,我们把该样本添加到正确预测列表。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "第一步，显示测试集中的图片一遍熟悉图片内容。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAB6CAYAAACvHqiXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztfWmQJVl13ncz8+2vXu1dXdV7T3fPDjMwDCAhNALJHpAE\nCluBkSV7bOOYCIcISw5FWMj6oeCfHHYorB82jgmBQEsIEYAERlgGhl1iYHpWmOl1eu+upWuvevvL\nvP5xzs1zXi091QtdXcX9Ijoq+2a+zHtv3sw853xnMdZaeHh4eHhsfQSb3QEPDw8Pj1sD/0L38PDw\n2CbwL3QPDw+PbQL/Qvfw8PDYJvAvdA8PD49tAv9C9/Dw8Ngm8C90Dw8Pj22Cm3qhG2MeN8acMMac\nNsZ85FZ1ysPDw8Pj+mFuNLDIGBMCOAngFwBcAvAsgF+z1r5667rn4eHh4bFRRDfx20cBnLbWngEA\nY8ynAbwfwLov9GKxaPv6+m7ikh4eHh4/eRgfH5+21g6/3nE380LfBeCi+v8lAG+91g/6+vrw5JNP\n3sQlPTw8PH7y8NGPfvT8Ro77sZOixpgnjTFHjTFHa7Xaj/tyHh4eHj+xuJkX+mUAe9T/d3NbF6y1\nT1lrH7HWPlIsFm/ich4eHh4e18LNvNCfBXDYGHPAGJMF8EEAX7w13fLw8PDwuF7csA3dWtsxxnwY\nwP8DEAL4hLX2les9z76FLwAAjE3StmyGumUC+d60Wk0AQCdu0zHZbLovTui3NhGPHRPEAIAgVH1u\nl2gfaF8m20j3hXDXlHPESQcA0O5I35LE8AUi7o9J9zV5n7QACY/LGGlttWgMcRytGnvAfWsl0lal\nbqDWitO20n1PQOPDH/5wut3pdFZd81bgus9nV/zVTYFuo9bANWrHK+PmL1HHu3mWk1zLW2utfrvj\nP/axj63at+9neW7jTto2c3UCANBsyJo5eNchAEBfbwUAkAmlP9kMLbysbuP1HBm1xjp1AEC5lOFz\nSF8j3g7VIp6bmwUA9PT0pG2ZTIbPS8eZQM7RSVoAgGAN0S0w0lirkjk0imhN5vP5dF+rRefo8DMI\nAIV8ga8lffvjP/qvXeffvWdHul0eOkK/C+W5rfSUAQBLTVnX1cUZ7i/d70QthogHUYhyaVs+5FeY\nem7TB5Cb4kTO79oS1eau4cZO1+e5XGPtGL5/JtDvhXiN4+i3uRz1NxtIv2Fp22Rl/mozxwAA33jm\nR6vOtVHcDCkKa+2XAXz5Zs7h4eHh4XFrcFMv9FuBFktZ1talkaXTHEppUwD6kkURS95a4uCvrslI\nY9NJFYl8ASOWAENuitQ5TEJSMzoihThpOVHnaBmSXOKQvrAtvS8O+FzytTYs5edV3yKWjIKIOh63\n26ojHR6SnMNJpGG4voUsDMN1990q3KjEr+cjlaOUFJk4kcryGKzscxqTgUhDcpabl9DXQrlI9zaw\n8ng0q9SWtITYz2fpvKUCHRepy7i1k1OLrJDl+67G0ozdcbSusmqduCmKIrm3TvIPlJTv5ibHWqte\nJtVam68pcNqthZw34ItlWEp1Uj8AtJtNHp8aC0uduMaaSKxI+Z2wn86VkWc6DklCDzJKQq8vU9/i\nKvdDzte0dFxbScYNnl8ltKPVJi0q4GeiXpN3i3tO9PicxhwE8hxap9nwZGqLQKcT8zFyTWPc+0nW\nTH8/jTlX6OHzyz1L3LrOST/i5TJuFj7038PDw2ObwL/QPTw8PLYJNt3kYtkkASumDstklIlFJUza\npAKFBTZrKLXVWRs0MZFllapjRaVJ2mHXcU51AgBjVxBzAAwTODYU1bEek243MUPqWbUlatTyMrWF\nVs7bk2dyTJF6lSIRSoUcjTMJWum+IDWvyNjdCNrJ+mYCbUL4cdWJ3ch5u8wb7vgu3dTt0iYimvNm\nm+Yj0np2TL8NzVrXTtZo2xiuNZaIzV6BMntlQ7pWJpC2XMDmNLdPEZrNOplmwlAReBHd93ZTiNUA\nbGLrUJs18kjGbFrKZgpyvJsHtcYcORyz2VDHe8xcvQoAGBnql+PZvBJm5VohX8vNs7L8IOLjm4ok\ndoRtuy1tKxFY2Rdzf2P1HMSGxpzvkX4M7huh3y7MAQDKteV0X6tB74i4LM9j0kuR5z1ZmXt33YDt\nsq2mPF/OgSKfl/uSTqlaE24du7+BsvF2eMyJXn58+Wwka7dQYOIYzmwoJp3EmXO1TH0LnBi8hO7h\n4eGxTbDpEnoUs2QeytcxYEkjF6qvv2Oc+EsZaOaHf9rREqwjebIi3ezcfzcAYHF+GgAwPSOSTCYi\naTyAfLlbHZqeupWAqGPnSeKxuUEAQDsUkqfFksPywmzadnmSJY28krzG5wEAe3fSNQd7tBTnXBll\n7E74iO1q1ygHLRnfCnfFWyLlp/1W2gO7dnaUeNNmTenUmTMAgJGd4u6WMLk9PCASZp6JpOQm+nit\nOcqyFJ50RLILWbrKKEIuw21BTOsom1FSX8iusUr7ygR0bxOjNLKE3XEbTI6q9dTgsReLsoZDx5Rq\n8ZDnocoulc8993y6q82aQn/lLWlbLsfOAWoKUtdZ1l4D5S5orHMOkDVpE0cMri+hdyCulQForSeh\nIoRZSwuVtlZidrNS5Hv8/LPpvtY0SeujD9wtfbtKz1zTyLyVeWBLdSJW82osOdbYg0EhIAMmRfUr\npVmk80Zt1lzaMllLJbovuYWFtC3acx8AoNbXm7YlrHXFfM/yiRCrqUUglrYwvnn52kvoHh4eHtsE\n/oXu4eHhsU2w6SYXp5ebSNLqOnW4oyMomYBqsRqcVWRTHDv1T5kk+Bzar/etP/8LAIDn/vF7AIAr\nbHoBgGrHRX6KKnb+0hQA4OwlSVGT6x8FAOweOUDXzIla2WJ1MVOWLJedBqmJM1NX0rZiP5lrLi1T\n9GFDqc8jPaQSFjOihsZtUpt1MNxKOnAtUvR2RIpe2zTD5FtGRfWyj3l9WUjw+QVSjSenyVRV6BH1\neZAjInVUoyMBdfToGp1d0YuNI8vmPavOkXGTH0u/Qzjyntoyyq+77dTtRM4RVmgejFVxB+zvnLho\n5FjW9fIimebKRSEBA55vHbUZcWT1PJOhs4tiSiywn3ZLWUZabbpWlNVrhtpijsTuKHOTi9LOKh9r\ny2s2idc3A+qZdybEQI097vBYla3DsEmkYei+ZxJZC2aITHG1Jelb++xJ6q8Rs1TC01V1/u3q+cq2\nOX7koiLleT60o0WDzadhg+dKLonmTupjfUJMqz2GnnnTOyTj4+u2A0c0q9gLnu9QkexRcPNmTi+h\ne3h4eGwTbLqE3gzoS7xQUxFkLN30l0WsqDDJFLGEogmr1O1IETSONK3V5tK2r3+J8sZMzpPEMbks\n37Pzl+m481ckxXuYJ2k9DitpW6lCX+JMkfZFeZEMcixF5gMZy3SLotRGd+9N2xpM1pw5QxL67LzK\nKbOLzrt/WDSFDLvuGeU2JvIZj1d9/W1yfTJpGpi5hoCgpfJgDQk9ZiksYWlER7O6CLyrM4tp22KV\nxlrX+TtqNJogR+RztS73tlxkiVT1zcn7G1VArldTyRnnYifz7cjQNV0OE45MVC6HEWuUkWIeQ0Pz\nYWN993h87AgQK9e25SWatwv6mpGLrBZpck+F5s25KL708svpvjfcfz8AINEulTHNb1679LKmUK+x\nBhzJ+TusIYaROAe0OV9Qs7l+SuxYSe8Jr2GrZUh2Ymhp90a+bu8Sz9XwSLqvsGMf9ccKGQl2vbRD\nO9OmeoZzs0xQXhgoF+AqP692ZDBtyyTUp4bS8EusJbaWaHxNnWOnwBG5Vbkv0SBpDyaj3DI5X0sP\n/zRUGkDH0NybQLno4uajvb2E7uHh4bFN4F/oHh4eHtsEm25yuVonNWO2LaTot/7hmwCA+46I6eLn\n7ieyoZ/91TUZ45LwBEp9iZl8UVwazp4nP+fZOqlCtjiQ7gvLTL4NiHmgwPVPWyplaouJuEo/9a1S\nlj5OTZAJZXFOkSWsEuYLYpq5MEdkbKZC6uTUuFSXKk8sAQB2VuT4gkvVmygybQWqNZ3cjFVOpWq6\n1MKhSvTktl06UJUTC0Gy+lvvoli1rWOZzQGOHC0o4qzBEXXjyuQyNUfbiSLM2mxPqS0RgTw1LfN3\n6fI4AOC+wwfTtrv276b+K7/8lJx1kb7ayuK6rcMUrkGVhmzyS9piTgjYxFdfkLGAzQ2WkzqFBRl7\nlu9VVs23aZOpLdZmCo6GNikRK+amapVMC5OTcnypUuZrqsRkPOetZTour/zhr84Tsfr8j8QMU8rR\nNQ8dlDmN2PTTrNH6K0QqkVST1las0kjH7lFrqPlYCTXFLoVt0hUrwvvUs5xhc1fu9Ck6/XPfSfd1\n3sKmKpWG1nKMSHZJno0GaB7KHO8R5uT4pETnN1YR9Zwcr2dQ3kGZy2yuWaY1mRkR5wdcpH1RRcyi\njas0v2FR2pIj5Jve4MRegSLxsx2anEjZEu01OP6NwkvoHh4eHtsEryuhG2M+AeCXAExZax/gtgEA\nfw1gP4BzAD5grZ1b7xzX7EAvSQm1Gfm2tLNEPM7WVPL3FrkRVbLs5qWIFCeRhqGQNo0WSbhXFf80\nvURf52IfESL9w0JUVhOSNIagovKYQGllRGpqVEmCaSzT8fsUuVJjaXyqJdKyYWlpYVZJZSyt1Pnr\nH2al35OLNI3jC6IV7BtiDeQaX/D5ugy0XCStIVB5JVyxji7B25E1Lgi3K23tGt/6NdwhJ8bJpXNg\ngLSdQl4kn2aDxlzMSdvOYdK0rBLfqjUaa4klmVZDpTvlQS83ZXydNM+GcqNL3SfdvlXD7JIYr+Vt\nmXcFDNRBTkLPKa2gzORzL5NZAbtfAkCO73FeC6SsRQUNWQtp0QMulNJalLXWU6J9/QOiSZ69RFrg\nmYsTadvJ008DAOamSSJdbsg5am2qORNBuSGy5P/g3UfStvf94uMAgF28npt5GWejWuXfyTUrXIDe\n1JewHjKhrD+X/tqRo4CkkI2UXFmeo2t1LpGbb0VpG0tX6PqtvERjWtB7wUxMpW2lMSY0K6x5Qp6l\nArvLZuel3w0mojvT42lbluews0hzlZsVx4h2nbWpgmg482fJmSJbEAm9Z5RIXJcKyioXxaYjw9Ua\nbiU3L6JvREL/JIDHV7R9BMDT1trDAJ7m/3t4eHh4bCJeV0K31n7bGLN/RfP7ATzG258C8E0Av3sj\nHbj7DY8CAC49cyJtK/fS1//Rt781bSuGZGdusYSspU/D2ehiK/k+enZQ/eoXXz4l5+0j6XDXPnLl\nssoWl2EpPGnOpG2tVrLqWiF/UV956SUAQEUlqC+WSDIoKTvalYlJAN15ZkKWOgbY3Wx+Tux/c7O0\nfXZcXLPGRsglK8qq6IYViCqiKcQsXbd1/T22TaZ/IXZNF6yiJVK7hg+jE+CVh2Qa4OLyfUC5jvax\n61e7rc7FUluxLDZJJ6EbDhYzykUsV3DuXaqsGhMjXTbHVX2Ta2a6D+Hd64voF8+d437LfC8t0rqL\n26IpXL5M2skcr4HqstiTdwySVF0uSVBQyMVZWipDYcS5hgLOJVRV0nvDDUYV2rhwhfiXs5eEZ6i2\n6Lf5XnadK8nEuJVYyorsNn6egnGuXJlM277znX8AANzLXMVwn0ik9WWS/F15OABo30v5VJYX1lfM\nc1kZu3XSeqJUZtZwAuVmu8yBgMuPvBEAUInenO6rLdE9aKu8TybHc6PKM2YKdN0qu2dqd9s250vJ\nqGejznOjnQbrbNevLdM1SwUZS4OPz5XlOR/ooXdPrN4Vy7x2wW6UhbbK2Mh90h7G7VuQP+lGbegj\n1lqnn0wAGLnWwR4eHh4eP37cNClqyXi57qfFGPOkMeaoMeaoztPs4eHh4XFrcaNui5PGmFFr7bgx\nZhTA1HoHWmufAvAUAIyNja168Rd7yVSw76AQNHW2QOw9cChtG2K1ff7sOQBAW0eXdch08eg7fyVt\n23vwEQDAgQfPpW3PvUBmkv4ymTCuTEkul4jdmHK6uAL3drkqZNf8LKmdA+WMPoT6wWaVoWHJ5eKK\nNkzPiQnFcDRlD7s8RqEiRljlfu3ipbRtuJ/U8sO7levUCnziz/5Czs/9yCj1r9xDKuOhA0IEv+UN\n5Fblyl5aZRZyJKPV9hWXY0eZVRxhl83R+TXZmc2SCWWwX7lPutqwqkZjmiMkQ+dodOT880wSz6tU\npUsLZAJoa1dNJjIH2fXs8CEhrDIumlAXhg+6DDBd+M4/PsPDVQVWHJFdl7VwboKIu7T2pxKP+nvJ\nZFFSJHGOj8soV8aIXeoCrilaU4RmxOewKm/RxCwR6W3Fbhd7nLsd5ztaVu6WfD8aDel3pYfO+7Y3\nP5i2VTnlc4NddC9cEFPKa6+9RmNXLnbnZ2ju6zU5b5QTch8ASiVxMOjwPLRjfc+40IwiAw2boAoj\nRHwuVmUsVxdo7Ea547a4ZmpWk4vz9BuXCyqXledgkdd4PqNefS6tsYoUbXL0Mrhm8EJd1qRLo1NU\n0bQ9u8nEG2ozYFoPl++VrmXh3hxqUSa3wG/xRiX0LwJ4grefAPCFm+6Jh4eHh8dNYSNui38FIkCH\njDGXAPwBgD8E8BljzIcAnAfwgRvtQJgjYuHK5LG07aE3UzL+Uq988cMlIqBilhIiVT7rzEUiLt7R\nf0BOXKTgk56SqtIe0bUK7CaYz6pS4fx13jU2mja9ypJJVpE7i0zMHNhDGsWRe+5L983OcjGLigQo\nXGF3KqNImL5+kmoXWPrU+U8KRfptfUn6feoCB3soYmtEUlfQ8TUV/FSn7YwK8lliAbeo2uJ77wEA\nNCyTR0pCz7GkpKVaV6hCZyHsHSBtJCWelLujc8MKlTTuIr20LJKwtHKOA78uT4nCNztDGlG9LpJd\n3GRJVOV8cTlFdu8hOmfvnt3pvlK6VjTpu76E/uIp6kexIBqRZY2w2ZH70stZMx3511JS8NVlugeh\nmquePGlknVhIcMMkYMi+bSaSQLVclSTLVlvI1tlZR4bqcmn0t8U5YpaqMlctdmfdMyyuj4P9tHhc\n4BIAzM5RHpjBPurHI2+8P913iV1TF+qyho9fovsSqHV9YAWTFqlMp4UeeuaWVUm5iFWaWGUZjDj4\nJuA1mSh3S8MFbyJ1TbfVbqkMk6xlRyx5a43IkaGx0gJdabuOWpWZApOW8eqsrS73S6ajNAX2GNAZ\nG/Oxy9DJ11JLzgXWdXsR33x21I14ufzaOrvefdNX9/Dw8PC4ZfCRoh4eHh7bBJueyyWTJ4Km0dDq\nM9dvVBGUxZIjmcgUoOuNliNSmT751MfTtl/+Fx+mc6jotizXUnTFMg4c3JXum5olgquxLGrzzh3k\nt64LBjS5zuPBQ0TY3nVIyNyFF6iWY3VJ1EpH6nRUhFydTSJ9XH8wthK11ttP6mJHVSQIAxrfpSti\nihh5A7rwgX/2z6WPTBaWVP4YR8IUlKnKpZZYXOT8Kh0xBWSYpIuU/61l1bWu/LNtQudzVdE1ERvx\n8ZmMjkBdbbZx/rcNzn9SUjky+jmfTtySvuVDGtf8jJgMLl0+BwA4xER6GCjTknUV7VWK4Wu4/C6y\nWc9q4pFjCwqhzMfuPXdR/12a4AlZa9NsKhoZkfqouSEyA1XnxZ874UjY3n6yV+RyEkvR4CHXOmJy\nyfNzELdljYVMLrqiL5msKrSRp+1H3yQmlCP7xuj8LVnrZ1+jcb124lUAwNvfIoTpnj10/IWXJedQ\nO3Y5ldavKZpV/chyTd3EipmzwCR4R6UpXuJI2ZiJz3yvmIpGSmwCU+ShW9faXBHC1Uylv7owx1qw\n/Gxqk0vMvu4uTXGgrpl1hh6VKKrJ7xSdOypik2MMzh+ji67wc6PrumrT643CS+geHh4e2wSbLqEb\njiCrKcm4wRJmRudxmGGXIs7XksF8um+0j76Yp45JVOiVS6dpoyal385fOgcAeHgnRafu2ifM4tgU\nSUjV0yKFDORIOuzpk7JSr712lq45RtL9/KJIT23+0k9eVRKYI0uUa2KNJXTDuR00FVJy2RsTifzM\nGpqP1vQE1kPSFgkilVDU/nKWzlvIy5zWOVNerU39OHfmnFyTSdG9B/albWcv0lx+6e+fTtvanOEy\nz/laiur8LrqutyJRh329JGU9/LCoGMNDJJXetZvmNFDugk7KcsQVIGRXfYdIb2OjdK/GdhGprTP4\n1di1rUtjuYYok2GifnjHWNqWZ0J6elrcSasctezC/RoqArR3mNbWLuV629NL46wMidQ+w0R6zBJb\nW1V0cy6SNUUkttqO8BSNJesyeuboHmesaFA7eO6H++Ue5JngG+4XFrPCrn0zFy4AAM6/di7dt3OA\n1v/C5DNpW4bJ8Fa4/iskUrlLQs4imVf5XeaniOCdXZYcKlfHaX77e2j9P3CfaAoZ1s6bihBus4ag\nCX23/l3Rl0AR9U5K1qUT45SI1axld24gnckV6TnkmYv4eL123W8yTnPSDzqfPlAumPE1XGk3Ci+h\ne3h4eGwT+Be6h4eHxzbBpptc0tS3Sn0ZHSJ1S6vvX3+ZfML7Ocn+4QFRgfI5JoUi8cW+OnWOTt+U\niLe9d5GfesjnLVaEgBoaIcJqZlbU2wUmQ3Vh8x07SF2O2BzUUOSlS7pUV+aBDv+4o07SaHJqzg59\nTweVCm641mDWyFhyTBrFtjsST+Nv/89X0u2EE/YHyoe3zARzjzJ/7D9MYx4eJBPD4KhEkQ5wn/Iq\nudT8MTJH/fCY1F2tW1dMg/4fKXW4wr89tFfMNm9/9E10rZL4eJdYbXcab0vNaYd9q2sLYmJrsx93\noSh96+sjc8MkJ0ObVkUyChyxOLJT5rlYVDEIK9DPJrZQmROaXMjDKBlodob6tLjIaZCViTDkCMPz\nlyUBVmWRzCW9vRKn4PzPm+wUYBRBmHPRjCW57wXrIkt1LmB6JkoFNkdaMcfsHqR5KSqCsrpI/e4o\nU44r/nGATUTHjp9J9x05Qom4oAjQK1fINz3fL2YvQG93k4Cu2EqizB9LHNNx9aqYEufn6LwnX/4B\nAOD4S99L9x06RDEf+w/dm7b1D7HZSJkrXKpoV+xEGzLC1Idd9S0t9CJtrkauFNJRpCsfr3n1NLJ6\nDbY9JV27kt/xWdX91u+SG4WX0D08PDy2CTZdQndRXL1lIaz6emjbqJwhi5Ykjek5+lIO9UjXS0zo\nxIFIJueunAMAjPRLMvx9/IV37mA/eE6iUy+PkyTfUxapPcNuVa+cvqB67CId6W9TfVWXOUKvTxUk\n6LDYOT6pEvD3UJ8ido0qFkUCc/lP0BZiNa5S30Z2rJ/L5dkXfpRuFzJEUDabQthmmdR769vekrad\nv0yS9gxzUg/cL65tWSY0a02R8jOs2bzpTUJoNjgSMcvS5OGDEq17P6dYHRsSibRSpHubKDfVixMU\npTg1x8U9pq+m+6pMls/Pi4Te4hS2GeWC6XLJuEjitiIoi300bw9Axtfbu/5cOkm7piJRQ+NK+IlW\nEHMq1ogjkBMr8lE2R+cfGpLI4zKv8bxyBe3lfkd8z7Q7p2XXwI5yJ+1ll85ARVcmnCY2ctGVTZG8\nezmBjO2I1hiz1tNSkY51vh9FXpvnJ2T9vfoaaX/NpkSgths0vzbU1Pv6cFJtPi9jv+duilQ+dK+4\nD9eWSFp/5XlyAX7hqBCx3/k2aYjHXpW1fuTehwAAh+8Wqb2vn9abI4vDrj66+V0j97ImW13JvM7q\nso8uejRWJGqSuk+uj6701MaVzZQ1rFNs3yi8hO7h4eGxTeBf6B4eHh7bBJtucnHRezt3iE+4qzGY\nKHJxdDep8kfZlDJvJEWtDUkt7x0S4rG3wj6geVGt97PJpcwpe//0E3+e7qvxtRbrQqbV2A9YZ9rc\nyZGcjVlS/6o5fU0yCx0/If7wk5NkPlhU0aN9fXTCSonU51CRWBmO3gtrl9O24RLt782LQqeSkAIA\nrl5U/vMDZDbavVtIwPvecJjOn5NzvPIiEU8jrAaXVTWjKa6vWKqIyWqwQse97/F3pm0BO3T39tJx\nQ4PiPz/LqYbPnpf5WJgnM9DigkTHLjH5PM9pimcXJQK0wwRvRqU1znKFoEBF1vVWaFx9HFnar8xT\nOTZpZQti2lquC+m8EoPsQ659+8tcfSZR6V8zAc3HDvZXNypKNss+084UBAB5jpYMVZ5dZ2JJqzQp\nk4vzwa9VZe24iMWcWpSWzS+1BZrvy+dkvmfZ+bmvIMePcIrhfF7X4GUTSkTmpqgo5PlVru+5Z1Se\nuR6u5rXYXJ/IS1RaXJfEywa6jfoWKt/0vkFKQ/uOx2jtHjokJrzvfuubAICzZ+XZqL7Az+2imOQe\nfANVO9qzh86l01PHHVrjsepbwqbdripdaf1c91d2uXq7miB31hLt8+4I0vRaXaQov+OU2UabcG4U\nXkL38PDw2CbYdAndkYCVfpHQOzF1KxeJG9gRLsxw9DmSvBYzEoGXGJL2RnbJl/7VY+Tu9FM/+2/T\ntu9x4YJqlaTEdksKXExNOFc8+cYtcw3ASEXl9Qckwe8q0DkWroo01AlJMh7ZIcRqzK5edSURNuok\nkVaZfOskIoG1GxQptyMjkuBYmSSpZkfaVkrol0++km4vMnH2y//kP6Rtjz9OyTG/9nVxb9zBZOGO\nIkeRKle4PEfPjfSKpNbD23nlLthhqcZJojpnzcQJkqQuTInrXosLlUR5SRPb00Mk8g6WGNut1URU\nRhUpcDkvdO6Lnh4aS6XSw/tUnUrOpzM5Kfe70Vi/elaRpdO2Im4L7ILZVxGtJ0lTOROhWVB1UlPS\nS0mHieU2LUe54iLuryLrOny/O7H0dXGGxqAf3AxL6MsLpA2OX5Ho6JEBGktfSaKdayxdJ0pT6PAZ\nHRG7iws2AMDdXGf0ofukaMjJM/S8vPBDcSxYCZ0yOuACFEEkWneGnQJiFV3p0s8GTBIfPiIEfMJu\nvuPjn0vb5qZprKeaotVNXqb6xHcdJtL13vvlHDtGiKSO1Lul0+biGyqlbsw1ct19XLMgSldOmdX7\n0xTNPA/6FGkxGSX6d0Wj3iC8hO7h4eGxTbCRAhd7APwZqBC0BfCUtfaPjTEDAP4awH4A5wB8wFq7\nfgnwdeCuZpu/AAAgAElEQVRyl/QPiQTR4a95I5DCCPkySxqcofDCRQlGeMdbyB2tsSxfzGIPuQmO\nX5bcG6dPUrXzjqsGrryZqmy37RkUN7OFBZKMessikd59hHJLPPvScQDA88fOSj9+7r0AurNEnjlN\nEvy8ytjoXB4bdZLM942IZFfgIJKBAZGMbUSSQ6e1vltTQ5UCe/CN1Md3vftdadtgH9m2f/qtyv7N\nkl0PawqVskjNIRdtcFXpAbHV6qIDC3Nkt62wxJOoDDIH734AALBjt2SknJ0jzaanT1wZXeY+Y1dX\nZHd2WFcaDQCW2aZsVckwVzjh4jjZ/p0WBABtLv6h87sUS+sHFlVZm+pRBS5ckNGUytOzyMFOCWdl\nPOQCcAD0cf6TMKOlT9rWWkyL65nVmDtpNKXfnRbNlVEFMWyTji8pjaWvjzScQpZs3JGRddLH2l1v\nj6zJFp+jprJJtjjDacCBLv1KMytyltJLiqdh4Rr33304bbuq3E3pXJoPYHu56luWdyf6QWTJ1dmY\nW0pb271nPwBg//79aduzk3S/O6o83tWpee4PSe/Hjr2c7nOBU3fdJf0eGSG3yZ4e4YvAAX6NFtvc\n1bOXYY1MBxE5t0UdV2SNdo2kUaWnTwtiCMJbUOBiIxJ6B8DvWGvvA/A2AL9pjLkPwEcAPG2tPQzg\naf6/h4eHh8cm4XVf6NbacWvt87y9BOAYgF0A3g/gU3zYpwD8ytpn8PDw8PC4HbguUtQYsx/AwwC+\nD2DEWuvyXk6ATDLXjYRrNPYOSFGDap3UnFosKoojwFytyJOvKFe4Gqk25ZLkIuHaAzh/UtTEy0wW\nvf3tlD5XpyXt4XS4A2PiJnVhlswq9aZKbl8i9bYyTKTRwz1Su/Iqq+Pnzr8oY6mReWJ+Qa61Y5hU\n415L/dlXFle/HRUuCmHEhOJSppaUCitOf4SD9zyUbn/wX/97Gl8savmJ00RMJkblwGHytM3q3+y8\nSlqTuDw2Qr+6wuoJhNhaWqSehJOkGl9R9UBdoZKkIWRTiQnYM6fEFHaWU7Y6t7+BIZkPZx5YWBDS\na2aaiEGrTCgBu8OZwOU1UZHHTMDmderg5ZW0siDHLpIz0zKW1+bomi7KEgD6+on8Hh2lpd9SUYXt\nFpltEit9XGSzWF2Zg2KO4AzZnKVrVzqzSr4kYymwu2JDrd2EicRSmd1g1TrJcpSkJpAdwdxQJKDh\n4xwp2VZFTC7NkCW1pmqQOlJx56is/5UIlckh3VbXhOH56nLnc78xq/a5KNOeHjEHpWRlV/ESZ8Kj\nay3NyX18gVNQv/LSs2nbwCDdx507hQjeObqfr0lmmEFlih3mgr5GEe/uPneUGbDDpGnqtqhdH9nc\nZZX5zSYrTTTXjw2TosaYMoDPAfhta+2i3mdpBtc08BpjnjTGHDXGHK3V1vcs8PDw8PC4OWxIQjeU\nAvBzAP7SWvt5bp40xoxaa8eNMaMAptb6rbX2KQBPAcDY2Niql/4SJxIpqEx1aea5RJVLYzJlaICk\nt5OBZIObmiXJZyaUL1xvmb6i9zwgRMeZcyQJuiICmqg8fJhIksMH7krbzo+TRPLKKz9M22amOUiF\niyD0K1e1S6+QRD8+Ld87w8RuqAKcRveQ+9c+/mLv7REJLM+lrJoNHfhAEpV2q1qJX/31f5lu9+8k\nqemlH4kU7MillpICYibpXKk1Tcq40l6xliC4LegSAzh3CmfBnJ4RF0XndqdiSdBX6eP+iKQ7O8Pa\nCEuJ09NCgDZZO+kot8+YywCGKpdLMU/znHMujboiu0veA5GeCiqL5ErMM9F75bK4/5WYrL5HFVxw\nGSmLnJ+mURetam6O3FvbbRlnjXOtFJXbZ2+F1n0pR38LiuyMWOqMFSna6bT4vCp7pyt/lhZjUEUT\nWMttqycvCpnUS5QrLWeTnLlKmsj0jLh4uqyIcyqfjtO0cj2iTa2EsVpCp7+aKDQs1eocJ6mkzX8d\nAQkA9WXqx8SEFMS4coW2F4pyXIbXkSP5Syp/TDGi4zRBfpmLapw6J++Uep2KuHRiOtfQsBQ7efBB\nClA8fEgk+uFhWguVXnHuyBVIk7Dg66tnr5MmcVTE9O0gRQ3llPw4gGPW2j9Su74I4AnefgLAF266\nNx4eHh4eN4yNSOg/DeBfAfihMcYZh/8LgD8E8BljzIcAnAfwgR9PFz08PDw8NoLXfaFba7+L9bNC\nvvtmO3DmNKk5ew9L+st8wGlAW0JcRaw2CTEiJGqZizbcc4/4AX/tK18GANQWxF+9OEjk1elLZB3a\ns1tI1AN3U+GFnFLjD+6l/fOz4l7/KtctTZhwuTQn5NEik7mNWMxHi/Nk1tmhCJfzM9Q2sIfMDzM5\n5ROdMImqzCs24lqKiajvK72oX3jxaLr98g/pu2sgphyXLyPSRRjSVLAZPkZU9YjT7ep0py6fSlb1\nN2A/9dDSvkpWomQDNku1Q2Ue4MhZ5TaMLOdaadfYP7oqJqsWk4amraJH2ebTUqR5zNGg1SU6vqju\n43Av9SNSpg5n2ViLGh0YpnXSrwqPuAINkZqPpWUiJpeXqb+5nJhLHKmo06+OjRAZnsuLecCRoZbz\niVQb0qMGE87zc5JfaGaWfL3ryrxzL6cpzrBvf3dBB653qtZTk2uhXkqjo8WHvMXmrFpVzr8wT6bH\nrIp6dWN/+utfT9ve+daH0QVVvCFx/uUdFaHJJhnlDg+TmoNoX6giZ196/jkAwPKc+LsPsn/9xXFp\nq7APfZafm0RFWFfK7A+v4gOyERcGyak4jIDNuHNkZjp3ViKx5+do3p4/qnL3cNzGnj0STTvGBWNG\nx+jZHxuR902J03Sbgqp3GqwfG7FR+EhRDw8Pj22CTc/l8uJpkpb3PvBo2paAvo5Gk4D8hV9kgmZ+\nXkibwQFy2Xvv4z+Xtj30Rsrj8JnP/03aZjgvQy9XX981Ji5XZSbrwo5IJgM7aXpGD4iUtcDFCZ5/\nkaTg8WXlLpUhArZ3VIiioUPU1lUYgd0ET3DRjtMTIsFmmT2qq8jIKk9DJxGp4j0iPAIAvvOtr6bb\nNc48l82o0mVFR8rKLQ8t5+9wVdIzWkKnfuRzirBlt7+sytIXlWis+SyNM6fyUbhUIUZliXTkdlsV\nzmgw4ZlKtTrCjo/Xpe3SEF8lEfeVaLu3RGMqF0QKzmXofBkj99Eo98OVaDNJp90cI3apjLuIPld+\nj+dPicZ5lsLrVRlnnTNM1pXPqdOEgoxzY5M1f+LYqwCA8+fOpW0uytkqd8ixUXIAGOCMl3XlTea2\n5+eE0Jxh0reuNGCXc8h5os0vipYU8NwXI1k7Ll/MxIRowCsl9LYqquFIedORc7ioVO2sZ0FtjkRd\nXpbJcsVU7j4i2vybHnoEAPDcy1L04plnKYvoPBdHiTtyD3aMErn5jne8I22L+D6fOy8uzs88Q7mg\nHriPotArveJcMcljnpwUBwC3dneOiHvjgQP76frsWFBdErdP52CQiUQraKyRw+h64SV0Dw8Pj20C\n/0L38PDw2CbYdJPLyQVS6adjlXo0Qyp40FIqSuJq8NHfsVGxOfzMTxGhmc+IGnpgH0V+/uKvfjBt\n++zf/B1da4LOO74gyl6jcRoAkIWovLN12j59XtRKsFpkh8mk0z8i5oe0rqCKxkzYPJEYMQG4ZFQL\nHMmZz6gkZJzCtmpUcikmI22iVbJu9WxkWKLnxutEEMWxqNkVrnMaqb4tThPZu7RY5X6Japo4dXmt\n6DVlVskU6D7YDF3fJVYDgIBtLkWVrMxVpo/bq81p4CRQJiu2izyTmwVl/hjoITV1j4oB2D1K/r+O\n92w2RFUPLK2nSEX29VVo3dUk11aKkycpJez999+XthXYhKKnI2D6MeHowEkVJeuSvTXryqzBJsRY\nmVUOHtoPABjeQf3XhRcybObpU4myHKGqy2Q6H/LjJyht7LIqiOH26RiGhE1K1SWZoxr3s8bRrC1l\nEnPFNC5MCvHoarzG16iDabsiQK3bSOGiPFUQKxJHpPKtKqh6uz/z2Lt5l/zAFa848pCYbB94M9XN\ndWVXA0UTuwIsBw9KvEnEc7r/sKTZHdtLRHOBI457lcnFjcsVcAHErLJjWNKAu2RfIZuqAsX+xuzg\n0FZ2usSsP5cbhZfQPTw8PLYJNl1CPzFP35QvfFeiMR/aR9LKzqwQBkWWEkZ30hdwdEiklrsOMrlp\nRaoY57wqn/j036Vtz71IJJOLRO0KvLSOlJJzxDm6RqyJPnYF7DDB2gkUaehmU5WSarT4vOpLHDFB\nGrI0ZlWukw5TRBn1NXelyFrt9SPJbFsk+t4SSRxLilhtxyS13XPvA/KbMZJWpjg6cEpFBy5zXhed\nrsFJljaW85YikkLueSOlJb2iSstdXSQNoN4SibHOhSV0VGqOXSlLrIn0qdwlw1zBfXRMJJ9Du8it\ncEdOxNRldnWcZbe+MCvzVywRCV5WEbmDnL/jylkhwhzaLN03lkXDCRwZqURMV7wiZtfEU6dOpvuW\nFhwxLY+YKwISKfE64ZDBgCNtoVwxB1mr0mRrjVMu1+sypxcvXuo6TgUfwrKLZ60l98xJ19Vp0YAz\n3E9X8q+jIimr7LbYUa6SEmm5vlRZV9pJyC6YkVURvPy8dlQEb4fnwZ1fl7FzAn9HaTiuHFxL5VAZ\n28v5mBJOUZuoIhL8nJ+9IK6g9ZbLA6QKpvQe6Lr+3IJcM2KJu1TZL4N1+ZAWZMxXJmf5HNTxnEoH\n7gJgTVnWR2Nu/bKIG4WX0D08PDy2CfwL3cPDw2ObYNNNLsushnzteVFXT75G0aPvebOQUneNkWp/\n9gxFar7zLWI6yLOqvtQSde4zf0/pMZ9/VRIs1VyUGps8ApWq1KlFgYpuc2aSWKlzTTaFtFklNMq3\nuckRl5oMiqLV9S+LnEgoC1eBPN2FmElFnRSrwwRitkeq/KzMhTZzRRJxxW1S3epKHa5dpMRkA6rC\n+jCnlc1wlZyCyqJVD10FFm2XWq1m1+pkpnknV426/15JXnXhApkzZuYl0rbpyDZFpkVMdBeYxRpS\nBGhfqcRXlnswMU1jOTEtSZoME1uVHWRGKlSEMC0yiarT8pYVybUSBb5nLWXWcGR1V51M53/O5opK\nRaKX8+zTXy4JqRfyuIoq2tSZOE4dp8RuC7NiCljgiM5Y+ZxnshyxqtZTjvV3w/NXU9GmU0zc1Zqi\nzoc8hv5eWU8tNs/V2Em+o5J/Jal5Red/5fkw68uE3/72N2QsHaoaVIpkPmJed21lVnHEvEtIpp+l\nNpu29PPoCMdGU9ritAIWp6JW9UMH+sicWy7rilk0Bs3vmnR8LuGZiujkMQfKhBJx0q/ArD7ODaEr\nvMLw+6MoxwcNNhcqwvt64SV0Dw8Pj22CTZfQB4cov8XsnHwexzmq7R+5bicAxO19vEVfwuGdEuVp\nQvoC/+CoRIv93dcp0quZiEQA/lIHwervWMySo1WfaeeOpqUEF+WZYcnA6M8p56HQpJerRalzz4R8\n/dCyxGGVpsBSvhbbR3eSNNlTUVJlrVtC3zk6kG5funCJx6SLCdD22ZMn0qYFdid0V68qt8gqS0NJ\n3MUc0/GqmECrSRLd89/9CgDgsZKM8wEeZ71XpGVHAuoo4AYTdgscvanJ2fPHKRpvui6Ri40MXb+w\nQ8bcv5MkrlyFxhSqSNEiu/3likKym3D9pe9cY+OO3AMXZZx0lLbGY3ekaEFFUgasNdZVTpTmLGmL\nF3RxCp4Hl0LW5csBhDzP5JVWwJdotWT+luZIIm80lvmvENnuTuXVmm/XOQWvqv/qCEz3V5ORzr2w\no7QTy1JtNrM+UZ9XkcrtkO+LSomdY6eDRLm6OrfNgK+pSeiE891orcBFzCZWRQHzqK2r22kUCc23\nL1B1caOQU1Y3JbI1JUh5eLpmaZs1Zq11uzVj1LOx8j3TUlGvls/RUK+PXEja1NjYPtwovITu4eHh\nsU2w6RK6k2YzKgtgp0HS1dlJkcqaVQr2eOebqIJ8oU9yJixwMYhvfV8yDtbZ9ttW2e5y7DbmpI+1\nKiiFSlpIP7bKtpZjyc44USlQx+dICimo8mfOxamtAmmWWGpzQRlNJQn29rPL5qgkyi+zP2RdBYKs\n/BTvPSKZ3BbZha96aVodwVn3lDvaLF83y2NuKXu52G1Xu6V1FSRgnHqZ8mdcXBLJZzig+ejScFhq\nWVb2+glLUuFptqleUjlAakXWcPZKgYGRAyTB5PvEdTW9Dyw1lcuiKRTZnh6oNWavYftd5DxBtSVx\nW5y6Qmuy0ZC+ufJxLo+HvsdO0wtUMFOGA98crwJIhsuIbe7aRbHNdmSdD6bZpLWzpNzj3G0rVdgd\nVkmGtk3z3FyWte6KZCwoidRJ5s4+bZS9PLGrg8tcbhuTrF90JVH3cblKPEox1PeA/sZqMbsAqBa7\n4XY6ypWPC3lYJY1LVkt5DjtsQ4+dNqjutQuq0sKztdTPZkPntom7jteau035nFi1uaBCXSSm+5ph\nS/ebc+f068I3tD0GL6F7eHh4/MTDv9A9PDw8tgle1+RijMkD+DaopkIE4LPW2j8wxgwA+GsA+wGc\nA/ABa+3ceudZDynJpInBkFTHliJtJpdJLXr+BBFL762JCrRkyRRxeU5MEnlWuTs1OUeDVUxXAzJS\nUXxuX5dbmnFuT3KcDbpTzmZy4oK2zK5eLZWC15lftNnBmViqHLFa7hPzSj/ngmiplJ/H2aUto9y1\n3rxCK6v0C0E4PEL5VcaVySVV/9RvmmxWcfUmtWtgfI0IwK49fOI2q+zVacn3EeQ4JbFymbvC13gR\noo6fjng+yqTGl/ZIkYzhMcrJM8hFJwAgx66ALdUTy2aBXMRV7iNNTLs2RVpewzds4hy50Ooq7E4F\nNzril9P3uurvWt3OsnlH57Fx+zXh2GETw/Iy13xt6pwr7DJntAshrYusKsYwsmuMz0ERnYtz8hh2\nuGCFVSS0M6fUWtoM48wZzscOq47PqLG7whO1mjIDrsDFi+KkcGqc+lFSNUIjthXFXSU5aE5dNGii\niPos5/rRbc5EE+vURjzPjrQ0KkeKI1u1bcvlg9H3xbnXJrGLIlVkJ5sou3I2uQIednVkq/tlW+WJ\nigdoXex6UFyze90tvYmULhuR0JsA3mWtfSOAhwA8box5G4CPAHjaWnsYwNP8fw8PDw+PTcJGStBZ\nAM7PKsP/LID3A3iM2z8F4JsAfve6e+DIBl04gINfEpX3weVTOTtFEsEnPvPldN+7HqMk92eviHRY\ndcEC6puVcZnqWEooKrejLBeuqC+JdO2IC6tIywwTlE4C1ESYkwQTRaDU2UVNt7nj+liqHlRJ8a/O\nUGDJ/LRkeJw/T8FUhw4ewHoo5EViy3EAS0blM4mZHNMf/04qufD49M5rSAldFBlLQ8s8vuNK6uvl\n8nTHG1II4BXWXmYqIrkO7qFxjR4gabxPuWDm2A0yUPk42rxWwkiVcmOJOEqDbOT4VLrWLmXXIEXD\nhF33lOto6l6oz8vaWmCdxCbnaLILZqct68lJ3LrivIMjzzNZXSKQywZqUpnXYj6n3P8K9JvZGbqm\nzqKYYY0z1NXlWRvtaGlyBanXFUjjCn4orWeZi6jUqpIPZiUCq8oXOmk1FqnWaQNdwUkhuy1a5xqo\nNC2WjFWcVTr3VrkmuhthxUcxhZPCtWtxh6/fVk4BCb+DrCsRqJ6HNC+T6ojB6rFYJr87HMBYUfmI\ndj9Izh2Rkfs9f5LzWe0WbfR6sSEbujEm5ALRUwC+aq39PoARa60L1ZsAsGYvjDFPGmOOGmOOruVV\n4uHh4eFxa7ChF7q1NrbWPgRgN4BHjTEPrNhvsY5MZ619ylr7iLX2kaLKbezh4eHhcWtxXX7o1tp5\nY8w3ADwOYNIYM2qtHTfGjGJlcpENYpArlTdUQYIqR7JlQ/Hndmk1nS/xt37wcrrvLNc3nK8KMzK7\nTGqz4hZRYvW9w2pXTlWvd6p6vqDyRATOR1hUe+cz22ETg9H+qayCxapCfYv9ZAsqf4dLsj8wRKaW\nliKEm1zQoZ6TayYcPagrwq9EW0V0VjkfR0+fXLNRJTVbF1CIWT1MM7aq1K1mtVUghVXpgS0TSlX2\nEf6OKkpyvkZtMypfRTRCFdBHdw+nbQeGaXuwl+YlUNGmVZYTGorYilj11zU/8xwFGnH19XxBhIcc\nz72OwrwWkjXyiLicNlaZfiyzyalJR53DRRrG2mTA60ivO7fGHEnbZfVK3HoSUjlm8rmVkXtb57S2\nztSSaAKUc780lHbsxmW1L7Y73pkrVD8iHottCZE9N0NmtHZr/TXZUX7oMR/XCjQh7PL66KIo3MTP\nUqDugUuRm2jTCJvFEpVu2hHSzvqhj3cmM23lSZx/uDKxOTNTaprR/uVsFoImbJ3ZRr0P2pzGeuBu\nKqaxa/+edF+D65G+dlxiZwpttmxLEPx143UldGPMsDGmj7cLAH4BwHEAXwTwBB/2BIAv3Hg3PDw8\nPDxuFhuR0EcBfMpQQoQAwGestV8yxnwPwGeMMR8CcB7AB26kAw2WOnPq09JkCSkTipTa4Q+lS9gf\nFESKO8dkaKBImw5LTx1FaDY4o1yVIzU18eOkplJWpLgCE6WBkioc4Vgo0vV1To2rnCkvUe5JERMi\n/RUhLXcOkFaycyeRf/NVkWQWOTPh8oJEKfZxoYPpqzrycwgabVXFPszS2PuH5ZrtMs1lp60y2yXu\nLxOmSkJ3Q9YRg6n0ptk/R9xxNsK2yqHS7KV+39Un9Er/AEV3liuy9MpFum85JpwbKl9Ki90crZKu\nQ+duqvvB2xnWtLTboiveoAk2ew3Wt8GufpF2V3WucNr1kcfuCl3o9bRS8uYOUFd1JCfPvXMbjFXk\nZZvnIVSaWZvzgcTKvbbUJM3GSeY6106zztL9GqXikjUifl0/Ij3f3O/ZSVHC2xyxqm/BKuihc86X\nICvXzLhsp3FXRQ7+Kc+VOp11GQqVhphnDaS/IkS6KznnCrLoOQ3ZxTSnNGCXp6UrOpbvi4ucXVpU\neVh4eSaRzNECp1KMhqQf+44Q8dnP0d+Xj59O902fpoyykepb/hp5cTaKjXi5vAzg4TXaZwC8+6Z7\n4OHh4eFxS+AjRT08PDy2CTY9OZdTCXMqiVHRESNtUTWdm2nCXtA6YVDC6lmnpUis2KXQ1MQWbSdp\nik75ns3NkqljVl2zwoURelUUZoV91/Mgc4yr3g0AEauEoap12eRkTq5Agj6uU+NajTWVxGh+hscu\nbG6eIxIb14huDJW61jdI5qBySfmhN9kEpUwundj5pjvfY5VojL/1QVc6UDYjqORSEavQRTZx9PSo\nCEYuIlDOCbldYt/0bE7U1RZvLrPffF0RvI64zSv1Nhs6n21Rm4MV5gx931tMemWzisTKrD+XLvo3\nUGaNjDP1aXMJ983NUFfR9jRyUCWvilcT0y5S2hW6aLXkvtfZ1BLXVUQnk6IlZZYq9JJK3+Fxthty\njmANm0jqj68JchcOwqaokorRqHJt2MVFMQM6i5VeMysRdtQcc93OREUIW1B/Q6iUwbwtUbWK0DS2\n6y8AJJx8rxZJIj+J9nbpr9V8czR3oy19c2vddPmyp53kM6lQVL6+JrwrnMp5+IjEigT8rjrx7Pfp\nmlNiMg35/ulCJWuZwK4XXkL38PDw2CYw9hZ8FTaKsbEx++STT96263l4eHhsB3z0ox99zlr7yOsd\n5yV0Dw8Pj20C/0L38PDw2CbwL3QPDw+PbQL/Qvfw8PDYJritpKgx5iqAKoDp1zv2DscQtvYYtnr/\nga0/hq3ef2Drj2Er9X+ftXb49Q66rS90ADDGHN0IW3snY6uPYav3H9j6Y9jq/Qe2/hi2ev/Xgje5\neHh4eGwT+Be6h4eHxzbBZrzQn9qEa95qbPUxbPX+A1t/DFu9/8DWH8NW7/8q3HYbuoeHh4fHjwfe\n5OLh4eGxTXBbX+jGmMeNMSeMMaeNMR+5nde+ERhj9hhjvmGMedUY84ox5re4fcAY81VjzCn+27/Z\nfb0WuMj3C8aYL/H/t1r/+4wxnzXGHDfGHDPGvH0LjuE/8Rr6kTHmr4wx+Tt5DMaYTxhjpowxP1Jt\n6/bXGPN7/FyfMMb8083pdTfWGcN/43X0sjHmb1w1Nt53x43henHbXuhc8eh/AngPgPsA/Jox5r7b\ndf0bRAfA71hr7wPwNgC/yX3+CICnrbWHATzN/7+T8VsAjqn/b7X+/zGAv7fW3gPgjaCxbJkxGGN2\nAfiPAB6x1j4AquXzQdzZY/gkqHawxpr95WfigwDu59/8L9OVi3bT8EmsHsNXATxgrX0DgJMAfg+4\no8dwXbidEvqjAE5ba89Ya1sAPg3g/bfx+tcNa+24tfZ53l4CvUh2gfr9KT7sUwB+ZXN6+PowxuwG\n8IsA/kQ1b6X+9wJ4J4CPA4C1tmWtnccWGgMjAlAwxkQAigCu4A4eg7X22wBmVzSv19/3A/i0tbZp\nrT0L4DToed9UrDUGa+1XrCSpfwZSkvmOHMP14na+0HcBuKj+f4nbtgSMMftBpfi+D2DEWjvOuyYA\njKzzszsB/wPAfwaQqLat1P8DAK4C+FM2G/2JMaaELTQGa+1lAP8dwAUA4wAWrLVfwRYaA2O9/m7V\nZ/vfAfi/vL1Vx9AFT4puAMaYMoDPAfhta+2i3mfJTeiOdBUyxvwSgClr7XPrHXMn958RAXgTgI9Z\nax8GpY7oMk3c6WNgW/P7QR+nMQAlY8xv6GPu9DGsxFbr70oYY34fZFL9y83uy63E7XyhXwawR/1/\nN7fd0TDGZEAv87+01n6emyeNMaO8fxTA1Hq/32T8NID3GWPOgUxc7zLG/AW2Tv8BkpQuWWu/z///\nLOgFv5XG8PMAzlprr1pr2wA+D+CnsLXGAKzf3y31bBtj/g2AXwLw61b8trfUGNbD7XyhPwvgsDHm\ngDEmCyIgvngbr3/dMFTY8OMAjllr/0jt+iKAJ3j7CQBfuN192wistb9nrd1trd0Pmu+vW2t/A1uk\n/zLRxI8AAAEWSURBVABgrZ0AcNEYczc3vRvAq9hCYwCZWt5mjCnymno3iI/ZSmMA1u/vFwF80BiT\nM8YcAHAYwA82oX+vC2PM4yAT5PustTW1a8uM4Zqw1t62fwDeC2KWXwPw+7fz2jfY33eA1MqXAbzI\n/94LYBDE8p8C8DUAA5vd1w2M5TEAX+LtLdV/AA8BOMr34W8B9G/BMXwUwHEAPwLw5wByd/IYAPwV\nyN7fBmlJH7pWfwH8Pj/XJwC8Z7P7f40xnAbZyt3z/L/v5DFc7z8fKerh4eGxTeBJUQ8PD49tAv9C\n9/Dw8Ngm8C90Dw8Pj20C/0L38PDw2CbwL3QPDw+PbQL/Qvfw8PDYJvAvdA8PD49tAv9C9/Dw8Ngm\n+P/jyitvLuJHGwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2955d7f2d68>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GroundTruth:    cat  ship  ship plane\n"
     ]
    }
   ],
   "source": [
    "dataiter = iter(testloader)\n",
    "images, labels = dataiter.next()\n",
    "# print images\n",
    "imshow(torchvision.utils.make_grid(images))\n",
    "print('GroundTruth: ', ' '.join('%5s' % classes[labels[j]] for j in range(4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在我们来看看神经网络认为以上图片是什么?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "outputs = net(images)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "输出是10个标签的概率。一个类别的概率越大,神经网络越认为他是这个类别。所以让我们得到最高概率的标签。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted:    cat  ship  ship plane\n"
     ]
    }
   ],
   "source": [
    "_, predicted = torch.max(outputs, 1)\n",
    "\n",
    "print('Predicted: ', ' '.join('%5s' % classes[predicted[j]]\n",
    "                              for j in range(4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这结果看起来非常的好。\n",
    "\n",
    "接下来让我们看看网络在整个测试集上的结果如何。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy of the network on the 10000 test images: 54 %\n"
     ]
    }
   ],
   "source": [
    "correct = 0\n",
    "total = 0\n",
    "with torch.no_grad():\n",
    "    for data in testloader:\n",
    "        images, labels = data\n",
    "        outputs = net(images)\n",
    "        _, predicted = torch.max(outputs.data, 1)\n",
    "        total += labels.size(0)\n",
    "        correct += (predicted == labels).sum().item()\n",
    "\n",
    "print('Accuracy of the network on the 10000 test images: %d %%' % (\n",
    "    100 * correct / total))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "结果看起来好于偶然，偶然的正确率为10%,似乎网络学习到了一些东西。\n",
    "\n",
    "那在什么类上预测较好，什么类预测结果不好呢？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy of plane : 52 %\n",
      "Accuracy of   car : 63 %\n",
      "Accuracy of  bird : 43 %\n",
      "Accuracy of   cat : 33 %\n",
      "Accuracy of  deer : 36 %\n",
      "Accuracy of   dog : 46 %\n",
      "Accuracy of  frog : 68 %\n",
      "Accuracy of horse : 62 %\n",
      "Accuracy of  ship : 80 %\n",
      "Accuracy of truck : 63 %\n"
     ]
    }
   ],
   "source": [
    "class_correct = list(0. for i in range(10))\n",
    "class_total = list(0. for i in range(10))\n",
    "with torch.no_grad():\n",
    "    for data in testloader:\n",
    "        images, labels = data\n",
    "        outputs = net(images)\n",
    "        _, predicted = torch.max(outputs, 1)\n",
    "        c = (predicted == labels).squeeze()\n",
    "        for i in range(4):\n",
    "            label = labels[i]\n",
    "            class_correct[label] += c[i].item()\n",
    "            class_total[label] += 1\n",
    "\n",
    "\n",
    "for i in range(10):\n",
    "    print('Accuracy of %5s : %2d %%' % (\n",
    "        classes[i], 100 * class_correct[i] / class_total[i]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来干什么?\n",
    "\n",
    "我们如何在GPU上运行神经网络呢?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 在GPU上训练\n",
    "你是如何把一个Tensor转换GPU上,你就如何把一个神经网络移动到GPU上训练。这个操作会递归遍历有所模块,并将其参数和缓冲区转换为CUDA张量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cuda:0\n"
     ]
    }
   ],
   "source": [
    "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n",
    "# Assume that we are on a CUDA machine, then this should print a CUDA device:\n",
    "#假设我们有一台CUDA的机器，这个操作将显示CUDA设备。\n",
    "print(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来假设我们有一台CUDA的机器，然后这些方法将递归遍历所有模块并将其参数和缓冲区转换为CUDA张量："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Net(\n",
       "  (conv1): Conv2d(3, 6, kernel_size=(5, 5), stride=(1, 1))\n",
       "  (pool): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
       "  (conv2): Conv2d(6, 16, kernel_size=(5, 5), stride=(1, 1))\n",
       "  (fc1): Linear(in_features=400, out_features=120, bias=True)\n",
       "  (fc2): Linear(in_features=120, out_features=84, bias=True)\n",
       "  (fc3): Linear(in_features=84, out_features=10, bias=True)\n",
       ")"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "net.to(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "请记住，你也必须在每一步中把你的输入和目标值转换到GPU上:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "inputs, labels = inputs.to(device), labels.to(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为什么我们没注意到GPU的速度提升很多?那是因为网络非常的小。\n",
    "\n",
    "#### 实践:\n",
    "尝试增加你的网络的宽度(第一个`nn.Conv2d`的第2个参数, 第二个`nn.Conv2d`的第一个参数,他们需要是相同的数字),看看你得到了什么样的加速。\n",
    "\n",
    "#### 实现的目标:\n",
    "* 深入了解了PyTorch的张量库和神经网络\n",
    "* 训练了一个小网络来分类图片"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 在多GPU上训练\n",
    "如果你希望使用所有GPU来更大的加快速度,请查看选读:[数据并行](https://pytorch.org/tutorials/beginner/blitz/data_parallel_tutorial.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来做什么?\n",
    "* 训练神经网络玩电子游戏\n",
    "* 在ImageNet上训练最好的ResNet\n",
    "* 使用对抗生成网络来训练一个人脸生成器\n",
    "* 使用LSTM网络训练一个字符级的语言模型\n",
    "* 更多示例\n",
    "* 更多教程\n",
    "* 在论坛上讨论PyTorch\n",
    "* 在Slack上与其他用户聊天"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "本章的官方代码：\n",
    "* Python：[cifar10_tutorial.py](download/cifar10_tutorial.py)\n",
    "* Jupyter notebook:[cifar10_tutorial.ipynb](download/cifar10_tutorial.ipynb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
